
!pip install ydata_profiling -U

Requirement already satisfied: ydata_profiling in /opt/conda/lib/python3.9/site-packages (4.1.2)
Requirement already satisfied: matplotlib<3.7,>=3.2 in /opt/conda/lib/python3.9/site-packages (from ydata_profiling) (3.3.4)
Requirement already satisfied: htmlmin==0.1.12 in /opt/conda/lib/python3.9/site-packages (from ydata_profiling) (0.1.12)
Requirement already satisfied: statsmodels<0.14,>=0.13.2 in /opt/conda/lib/python3.9/site-packages (from ydata_profiling) (0.13.2)
Requirement already satisfied: PyYAML<6.1,>=5.0.0 in /opt/conda/lib/python3.9/site-packages (from ydata_profiling) (6.0)
Requirement already satisfied: pandas!=1.4.0,<1.6,>1.1 in /opt/conda/lib/python3.9/site-packages (from ydata_profiling) (1.2.4)
Requirement already satisfied: multimethod<1.10,>=1.4 in /opt/conda/lib/python3.9/site-packages (from ydata_profiling) (1.9.1)
Requirement already satisfied: visions[type_image_path]==0.7.5 in /opt/conda/lib/python3.9/site-packages (from ydata_profiling) (0.7.5)
Requirement already satisfied: jinja2<3.2,>=2.11.1 in /opt/conda/lib/python3.9/site-packages (from ydata_profiling) (3.0.1)
Requirement already satisfied: numpy<1.24,>=1.16.0 in /opt/conda/lib/python3.9/site-packages (from ydata_profiling) (1.21.1)
Requirement already satisfied: typeguard<2.14,>=2.13.2 in /opt/conda/lib/python3.9/site-packages (from ydata_profiling) (2.13.3)
Requirement already satisfied: scipy<1.10,>=1.4.1 in /opt/conda/lib/python3.9/site-packages (from ydata_profiling) (1.9.1)
Requirement already satisfied: seaborn<0.13,>=0.10.1 in /opt/conda/lib/python3.9/site-packages (from ydata_profiling) (0.11.1)
Requirement already satisfied: requests<2.29,>=2.24.0 in /opt/conda/lib/python3.9/site-packages (from ydata_profiling) (2.25.1)
Requirement already satisfied: imagehash==4.3.1 in /opt/conda/lib/python3.9/site-packages (from ydata_profiling) (4.3.1)
Requirement already satisfied: tqdm<4.65,>=4.48.2 in /opt/conda/lib/python3.9/site-packages (from ydata_profiling) (4.61.2)
Requirement already satisfied: pydantic<1.11,>=1.8.1 in /opt/conda/lib/python3.9/site-packages (from ydata_profiling) (1.8.2)
Requirement already satisfied: phik<0.13,>=0.11.1 in /opt/conda/lib/python3.9/site-packages (from ydata_profiling) (0.12.3)
Requirement already satisfied: pillow in /opt/conda/lib/python3.9/site-packages (from imagehash==4.3.1->ydata_profiling) (8.4.0)
Requirement already satisfied: PyWavelets in /opt/conda/lib/python3.9/site-packages (from imagehash==4.3.1->ydata_profiling) (1.4.1)
Requirement already satisfied: tangled-up-in-unicode>=0.0.4 in /opt/conda/lib/python3.9/site-packages (from visions[type_image_path]==0.7.5->ydata_profiling) (0.2.0)
Requirement already satisfied: networkx>=2.4 in /opt/conda/lib/python3.9/site-packages (from visions[type_image_path]==0.7.5->ydata_profiling) (3.1)
Requirement already satisfied: attrs>=19.3.0 in /opt/conda/lib/python3.9/site-packages (from visions[type_image_path]==0.7.5->ydata_profiling) (21.2.0)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.9/site-packages (from jinja2<3.2,>=2.11.1->ydata_profiling) (2.1.1)
Requirement already satisfied: cycler>=0.10 in /opt/conda/lib/python3.9/site-packages (from matplotlib<3.7,>=3.2->ydata_profiling) (0.11.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3 in /opt/conda/lib/python3.9/site-packages (from matplotlib<3.7,>=3.2->ydata_profiling) (2.4.7)
Requirement already satisfied: python-dateutil>=2.1 in /opt/conda/lib/python3.9/site-packages (from matplotlib<3.7,>=3.2->ydata_profiling) (2.8.1)
Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/lib/python3.9/site-packages (from matplotlib<3.7,>=3.2->ydata_profiling) (1.4.4)
Requirement already satisfied: pytz>=2017.3 in /opt/conda/lib/python3.9/site-packages (from pandas!=1.4.0,<1.6,>1.1->ydata_profiling) (2021.1)
Requirement already satisfied: joblib>=0.14.1 in /opt/conda/lib/python3.9/site-packages (from phik<0.13,>=0.11.1->ydata_profiling) (1.1.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.9/site-packages (from pydantic<1.11,>=1.8.1->ydata_profiling) (4.3.0)
Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.9/site-packages (from python-dateutil>=2.1->matplotlib<3.7,>=3.2->ydata_profiling) (1.16.0)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.9/site-packages (from requests<2.29,>=2.24.0->ydata_profiling) (2022.6.15)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /opt/conda/lib/python3.9/site-packages (from requests<2.29,>=2.24.0->ydata_profiling) (1.26.6)
Requirement already satisfied: idna<3,>=2.5 in /opt/conda/lib/python3.9/site-packages (from requests<2.29,>=2.24.0->ydata_profiling) (2.10)
Requirement already satisfied: chardet<5,>=3.0.2 in /opt/conda/lib/python3.9/site-packages (from requests<2.29,>=2.24.0->ydata_profiling) (4.0.0)
Requirement already satisfied: packaging>=21.3 in /opt/conda/lib/python3.9/site-packages (from statsmodels<0.14,>=0.13.2->ydata_profiling) (21.3)
Requirement already satisfied: patsy>=0.5.2 in /opt/conda/lib/python3.9/site-packages (from statsmodels<0.14,>=0.13.2->ydata_profiling) (0.5.2)


import pandas as pd 

# для анализа данных
from ydata_profiling import ProfileReport


import numpy as np

# для графиков
import matplotlib.pyplot as plt
import seaborn as sns

# для скрытия ошибок
import warnings
from IPython.display import display
#для загрузки данных и с сервера и локально
import os

# для условно рандомных состояний
from numpy.random import RandomState
# метрика MAE
from sklearn.metrics import mean_absolute_error
# для создания собственных метрик
from sklearn.metrics import make_scorer
# для подбора параметров моделей
from sklearn.model_selection import GridSearchCV, KFold
# для расчета метрик при кросс валидации
from sklearn.model_selection import cross_val_score
# Линейная регрессия
from sklearn.linear_model import LinearRegression
# Ridge регрессия
from sklearn.linear_model import Ridge
# Lasso регрессия
from sklearn.linear_model import Lasso
# ElasticNet регрессия
from sklearn.linear_model import ElasticNet
# регрессор Дерево Решений
from sklearn.tree import DecisionTreeRegressor
# регрессор Случайный Лес
from sklearn.ensemble import RandomForestRegressor
# регрессор для проверки адекватности
from sklearn.dummy import DummyRegressor
# для создания конвеера/ трудопровода
from sklearn.pipeline import Pipeline, make_pipeline
# для маштабирования признаков
from sklearn.preprocessing import StandardScaler
# заполнение пропусков
from sklearn.impute import SimpleImputer


# для скрытия ошибок
warnings.filterwarnings("ignore")

# для увеличения окна вывода 
pd.options.display.max_rows = 300

# для задания размера графиков по умолчанию
plt.rcParams["figure.figsize"] = (7, 5)


#путь с сервероной версии
pth1 = '/datasets/gold_recovery_train_new.csv'
# моя локальная версия
pth2 = 'gold_recovery_train_new.csv'

# загрузка данных
if os.path.exists(pth1):
    df_golden_recovery_train = pd.read_csv(pth1, parse_dates=['date'], index_col='date')
elif os.path.exists(pth2):
    df_golden_recovery_train = pd.read_csv(pth2, parse_dates=['date'], index_col='date')
else:
    display('Проверьте пути к файлам с данными!')

df_golden_recovery_train.head()


#путь с сервероной версии
pth1 = '/datasets/gold_recovery_test_new.csv'
# моя локальная версия
pth2 = 'gold_recovery_test_new.csv'

# загрузка данных
if os.path.exists(pth1):
    df_golden_recovery_test = pd.read_csv(pth1, parse_dates=['date'], index_col='date')
elif os.path.exists(pth2):
    df_golden_recovery_test = pd.read_csv(pth2, parse_dates=['date'], index_col='date')
else:
    display('Проверьте пути к файлам с данными!')

df_golden_recovery_test.head()


#путь с сервероной версии
pth1 = '/datasets/gold_recovery_full_new.csv'
# моя локальная версия
pth2 = 'gold_recovery_full_new.csv'

# загрузка данных
if os.path.exists(pth1):
    df_golden_recovery_full = pd.read_csv(pth1, parse_dates=['date'], index_col='date')
elif os.path.exists(pth2):
    df_golden_recovery_full = pd.read_csv(pth2, parse_dates=['date'], index_col='date')
else:
    display('Проверьте пути к файлам с данными!')

df_golden_recovery_full.head()


profile_train = ProfileReport(df_golden_recovery_train, minimal=True)
profile_train.to_notebook_iframe()

Summarize dataset:   0%|          | 0/5 [00:00<?, ?it/s]

Generate report structure:   0%|          | 0/1 [00:00<?, ?it/s]

Render HTML:   0%|          | 0/1 [00:00<?, ?it/s]


profile_test = ProfileReport(df_golden_recovery_test, minimal=True)
profile_test.to_notebook_iframe()

Summarize dataset:   0%|          | 0/5 [00:00<?, ?it/s]

Generate report structure:   0%|          | 0/1 [00:00<?, ?it/s]

Render HTML:   0%|          | 0/1 [00:00<?, ?it/s]


profile_full = ProfileReport(df_golden_recovery_full, minimal=True)
profile_full.to_notebook_iframe()

Summarize dataset:   0%|          | 0/5 [00:00<?, ?it/s]

Generate report structure:   0%|          | 0/1 [00:00<?, ?it/s]

Render HTML:   0%|          | 0/1 [00:00<?, ?it/s]


(df_golden_recovery_full.columns != df_golden_recovery_train.columns).sum()

0


# запишем и колонки, что в тесте
columns_in_test = df_golden_recovery_test.columns.to_list()
columns_in_test

['primary_cleaner.input.sulfate',
 'primary_cleaner.input.depressant',
 'primary_cleaner.input.feed_size',
 'primary_cleaner.input.xanthate',
 'primary_cleaner.state.floatbank8_a_air',
 'primary_cleaner.state.floatbank8_a_level',
 'primary_cleaner.state.floatbank8_b_air',
 'primary_cleaner.state.floatbank8_b_level',
 'primary_cleaner.state.floatbank8_c_air',
 'primary_cleaner.state.floatbank8_c_level',
 'primary_cleaner.state.floatbank8_d_air',
 'primary_cleaner.state.floatbank8_d_level',
 'rougher.input.feed_ag',
 'rougher.input.feed_pb',
 'rougher.input.feed_rate',
 'rougher.input.feed_size',
 'rougher.input.feed_sol',
 'rougher.input.feed_au',
 'rougher.input.floatbank10_sulfate',
 'rougher.input.floatbank10_xanthate',
 'rougher.input.floatbank11_sulfate',
 'rougher.input.floatbank11_xanthate',
 'rougher.state.floatbank10_a_air',
 'rougher.state.floatbank10_a_level',
 'rougher.state.floatbank10_b_air',
 'rougher.state.floatbank10_b_level',
 'rougher.state.floatbank10_c_air',
 'rougher.state.floatbank10_c_level',
 'rougher.state.floatbank10_d_air',
 'rougher.state.floatbank10_d_level',
 'rougher.state.floatbank10_e_air',
 'rougher.state.floatbank10_e_level',
 'rougher.state.floatbank10_f_air',
 'rougher.state.floatbank10_f_level',
 'secondary_cleaner.state.floatbank2_a_air',
 'secondary_cleaner.state.floatbank2_a_level',
 'secondary_cleaner.state.floatbank2_b_air',
 'secondary_cleaner.state.floatbank2_b_level',
 'secondary_cleaner.state.floatbank3_a_air',
 'secondary_cleaner.state.floatbank3_a_level',
 'secondary_cleaner.state.floatbank3_b_air',
 'secondary_cleaner.state.floatbank3_b_level',
 'secondary_cleaner.state.floatbank4_a_air',
 'secondary_cleaner.state.floatbank4_a_level',
 'secondary_cleaner.state.floatbank4_b_air',
 'secondary_cleaner.state.floatbank4_b_level',
 'secondary_cleaner.state.floatbank5_a_air',
 'secondary_cleaner.state.floatbank5_a_level',
 'secondary_cleaner.state.floatbank5_b_air',
 'secondary_cleaner.state.floatbank5_b_level',
 'secondary_cleaner.state.floatbank6_a_air',
 'secondary_cleaner.state.floatbank6_a_level']


# columns_not_in_test = df_golden_recovery_train.columns[~df_golden_recovery_train.columns.isin(df_golden_recovery_test.columns)].to_list()
columns_not_in_test = df_golden_recovery_train.columns.difference(df_golden_recovery_test.columns).to_list()
columns_not_in_test

['final.output.concentrate_ag',
 'final.output.concentrate_au',
 'final.output.concentrate_pb',
 'final.output.concentrate_sol',
 'final.output.recovery',
 'final.output.tail_ag',
 'final.output.tail_au',
 'final.output.tail_pb',
 'final.output.tail_sol',
 'primary_cleaner.output.concentrate_ag',
 'primary_cleaner.output.concentrate_au',
 'primary_cleaner.output.concentrate_pb',
 'primary_cleaner.output.concentrate_sol',
 'primary_cleaner.output.tail_ag',
 'primary_cleaner.output.tail_au',
 'primary_cleaner.output.tail_pb',
 'primary_cleaner.output.tail_sol',
 'rougher.calculation.au_pb_ratio',
 'rougher.calculation.floatbank10_sulfate_to_au_feed',
 'rougher.calculation.floatbank11_sulfate_to_au_feed',
 'rougher.calculation.sulfate_to_au_concentrate',
 'rougher.output.concentrate_ag',
 'rougher.output.concentrate_au',
 'rougher.output.concentrate_pb',
 'rougher.output.concentrate_sol',
 'rougher.output.recovery',
 'rougher.output.tail_ag',
 'rougher.output.tail_au',
 'rougher.output.tail_pb',
 'rougher.output.tail_sol',
 'secondary_cleaner.output.tail_ag',
 'secondary_cleaner.output.tail_au',
 'secondary_cleaner.output.tail_pb',
 'secondary_cleaner.output.tail_sol']


len(columns_not_in_test)

34


df_golden_recovery_full.loc[df_golden_recovery_test.index, columns_not_in_test].isna().sum().sum()

291


# из test + из объектов test и не включенных колонок + из train
90 + 291 + 4100

4481


# df_golden_recovery_train.columns
df_golden_recovery_train['rougher.output.recovery_new'] = (df_golden_recovery_train['rougher.output.concentrate_au']  
     * (df_golden_recovery_train['rougher.input.feed_au'] - df_golden_recovery_train['rougher.output.tail_au']) 
     / df_golden_recovery_train['rougher.input.feed_au'] 
     / (df_golden_recovery_train['rougher.output.concentrate_au'] - df_golden_recovery_train['rougher.output.tail_au']) 
     * 100)

df_golden_recovery_train['rougher.output.recovery_new'].describe()

count    14149.000000
mean        82.704502
std         14.479156
min         -0.000000
25%         79.993067
50%         85.299462
75%         90.165021
max        100.000000
Name: rougher.output.recovery_new, dtype: float64


mean_absolute_error(df_golden_recovery_train['rougher.output.recovery_new']
                    , df_golden_recovery_train['rougher.output.recovery'] )

1.0748911125799084e-14


# df_golden_recovery_train.columns
df_golden_recovery_train['final.output.recovery_new'] = (df_golden_recovery_train['final.output.concentrate_au']  
     * (df_golden_recovery_train['rougher.input.feed_au'] - df_golden_recovery_train['final.output.tail_au']) 
     / df_golden_recovery_train['rougher.input.feed_au'] 
     / (df_golden_recovery_train['final.output.concentrate_au'] - df_golden_recovery_train['final.output.tail_au']) 
     * 100)


mean_absolute_error(df_golden_recovery_train['final.output.recovery_new']
                   , df_golden_recovery_train['final.output.recovery'])

9.197645354602618e-15


df_golden_recovery_full.columns[df_golden_recovery_full.columns.str.contains('au')]

Index(['final.output.concentrate_au', 'final.output.tail_au',
       'primary_cleaner.output.concentrate_au',
       'primary_cleaner.output.tail_au',
       'rougher.calculation.sulfate_to_au_concentrate',
       'rougher.calculation.floatbank10_sulfate_to_au_feed',
       'rougher.calculation.floatbank11_sulfate_to_au_feed',
       'rougher.calculation.au_pb_ratio', 'rougher.input.feed_au',
       'rougher.output.concentrate_au', 'rougher.output.tail_au',
       'secondary_cleaner.output.tail_au'],
      dtype='object')


au_concentrate_columns = ['rougher.input.feed_au', 'rougher.output.concentrate_au'
                          , 'primary_cleaner.output.concentrate_au'
                          , 'final.output.concentrate_au'
                         ]


df_golden_recovery_full[au_concentrate_columns].mean()

rougher.input.feed_au                     8.266097
rougher.output.concentrate_au            19.772000
primary_cleaner.output.concentrate_au    32.119400
final.output.concentrate_au              44.076513
dtype: float64


df_golden_recovery_full.columns[df_golden_recovery_full.columns.str.contains('ag')]

Index(['final.output.concentrate_ag', 'final.output.tail_ag',
       'primary_cleaner.output.concentrate_ag',
       'primary_cleaner.output.tail_ag', 'rougher.input.feed_ag',
       'rougher.output.concentrate_ag', 'rougher.output.tail_ag',
       'secondary_cleaner.output.tail_ag'],
      dtype='object')


ag_concentrate_columns = ['rougher.input.feed_ag', 'rougher.output.concentrate_ag'
                          , 'primary_cleaner.output.concentrate_ag'
                          , 'final.output.concentrate_ag'
                         ]


df_golden_recovery_full[ag_concentrate_columns].mean()

rougher.input.feed_ag                     8.794927
rougher.output.concentrate_ag            11.994759
primary_cleaner.output.concentrate_ag     8.442408
final.output.concentrate_ag               5.168470
dtype: float64


df_golden_recovery_full.columns[df_golden_recovery_full.columns.str.contains('pb')]

Index(['final.output.concentrate_pb', 'final.output.tail_pb',
       'primary_cleaner.output.concentrate_pb',
       'primary_cleaner.output.tail_pb', 'rougher.calculation.au_pb_ratio',
       'rougher.input.feed_pb', 'rougher.output.concentrate_pb',
       'rougher.output.tail_pb', 'secondary_cleaner.output.tail_pb'],
      dtype='object')


pb_concentrate_columns = ['rougher.input.feed_pb', 'rougher.output.concentrate_pb'
                          , 'primary_cleaner.output.concentrate_pb'
                          , 'final.output.concentrate_pb'
                         ]


df_golden_recovery_full[pb_concentrate_columns].mean()

rougher.input.feed_pb                    3.598085
rougher.output.concentrate_pb            7.606272
primary_cleaner.output.concentrate_pb    9.829530
final.output.concentrate_pb              9.978895
dtype: float64


df_golden_recovery_full.columns[df_golden_recovery_full.columns.str.contains('sol')]

Index(['final.output.concentrate_sol', 'final.output.tail_sol',
       'primary_cleaner.output.concentrate_sol',
       'primary_cleaner.output.tail_sol', 'rougher.input.feed_sol',
       'rougher.output.concentrate_sol', 'rougher.output.tail_sol',
       'secondary_cleaner.output.tail_sol'],
      dtype='object')


sol_concentrate_columns = ['rougher.input.feed_sol', 'rougher.output.concentrate_sol'
                          , 'primary_cleaner.output.concentrate_sol'
                          , 'final.output.concentrate_sol'
                         ]


df_golden_recovery_full[sol_concentrate_columns].mean()

rougher.input.feed_sol                    36.695114
rougher.output.concentrate_sol            28.805571
primary_cleaner.output.concentrate_sol    10.489499
final.output.concentrate_sol               9.501224
dtype: float64


fig, ax = plt.subplots(4, figsize=(10, 24))
fig.subplots_adjust(hspace=0.5)

# График для золота
x_0 = df_golden_recovery_full[au_concentrate_columns]
ax[0].set_xlabel('Концентрация золота, %')
ax[0].set_ylabel('Количество образцов, шт.')
ax[0].set_title('Концентрация Au (золота) по этапам очистки')
ax[0].grid(True)
ax[0].hist(x_0, bins=50)
ax[0].legend(['Золотоносная руда', 'Черновой концентрат', 'Концентрат после первой очиски', 'Финальный концентрат'])

# График для серебра
x_1 = df_golden_recovery_full[ag_concentrate_columns]
ax[1].set_xlabel('Концентрация серебра, %')
ax[1].set_ylabel('Количество образцов, шт.')
ax[1].set_title('Концентрация Ag (серебра) по этапам очистки')
ax[1].grid(True)
ax[1].hist(x_1, bins=50)
ax[1].legend(['Золотоносная руда', 'Черновой концентрат', 'Концентрат после первой очиски', 'Финальный концентрат'])

# График для свинца
x_2 = df_golden_recovery_full[pb_concentrate_columns]
ax[2].set_xlabel('Концентрация свинца, %')
ax[2].set_ylabel('Количество образцов, шт.')
ax[2].set_title('Концентрация Pb (свинца) по этапам очистки')
ax[2].grid(True)
ax[2].hist(x_2, bins=50)
ax[2].legend(['Золотоносная руда', 'Черновой концентрат', 'Концентрат после первой очиски', 'Финальный концентрат'])

# График для солей
x_3 = df_golden_recovery_full[sol_concentrate_columns]
ax[3].set_xlabel('Концентрация солей, %')
ax[3].set_ylabel('Количество образцов, шт.')
ax[3].set_title('Концентрация Sol (солей) по этапам очистки')
ax[3].grid(True)
ax[3].hist(x_3, bins=50)
ax[3].legend(['Золотоносная руда', 'Черновой концентрат', 'Концентрат после первой очиски', 'Финальный концентрат'])

plt.show()


bx = (df_golden_recovery_full[au_concentrate_columns+ag_concentrate_columns+pb_concentrate_columns+sol_concentrate_columns]
      .boxplot(rot=90)
     )
bx.set_xlabel('Этап, Элемент')
bx.set_ylabel('Концентрация в %')
bx.set_title('Ящик с усами для концентрации элементов на каждом этапе')
plt.show()


df_golden_recovery_full[au_concentrate_columns+ag_concentrate_columns+pb_concentrate_columns+sol_concentrate_columns].corr()


ax = (df_golden_recovery_full[au_concentrate_columns+ag_concentrate_columns+pb_concentrate_columns+sol_concentrate_columns]
    .mean().plot(kind='bar'
                 , legend=True
                 , grid=True
                 , label='full'
                 , alpha = 0.3
                 , title='Средняя концентрации Au, Ag, Pb, Sol на всех этапах'
                 , xlabel='Этапы очистки, элемент'
                 , ylabel='Концентрация, %'
                )
)
(
df_golden_recovery_train[au_concentrate_columns+ag_concentrate_columns+pb_concentrate_columns+sol_concentrate_columns]
    .mean().plot(kind='bar'
                 , alpha=0.2
                 , label='train'
                 , legend=True
                 , ax=ax
                 , color='r'
                )
)
plt.show()


df_golden_recovery_full['final.output.tail_au'].describe()

count    19439.000000
mean         3.042467
std          0.922808
min          0.000000
25%          2.461138
50%          2.984909
75%          3.571351
max          8.245022
Name: final.output.tail_au, dtype: float64


df_golden_recovery_full['rougher.input.feed_total'] = (df_golden_recovery_full['rougher.input.feed_au']
                                                       + df_golden_recovery_full['rougher.input.feed_ag']
                                                       + df_golden_recovery_full['rougher.input.feed_pb']
                                                       + df_golden_recovery_full['rougher.input.feed_sol']
                                                      )


df_golden_recovery_full['rougher.output.concentrate_total'] = (df_golden_recovery_full['rougher.output.concentrate_au']
                                                       + df_golden_recovery_full['rougher.output.concentrate_ag']
                                                       + df_golden_recovery_full['rougher.output.concentrate_pb']
                                                       + df_golden_recovery_full['rougher.output.concentrate_sol']
                                                      )


df_golden_recovery_full['primary_cleaner.output.concentrate_total'] = (df_golden_recovery_full['primary_cleaner.output.concentrate_au']
                                                       + df_golden_recovery_full['primary_cleaner.output.concentrate_ag']
                                                       + df_golden_recovery_full['primary_cleaner.output.concentrate_pb']
                                                       + df_golden_recovery_full['primary_cleaner.output.concentrate_sol']
                                                      )


df_golden_recovery_full['final.output.concentrate_total'] = (df_golden_recovery_full['final.output.concentrate_au']
                                                       + df_golden_recovery_full['final.output.concentrate_ag']
                                                       + df_golden_recovery_full['final.output.concentrate_pb']
                                                       + df_golden_recovery_full['final.output.concentrate_sol']
                                                      )


df_golden_recovery_full['rougher.input.feed_total'].describe()

count    19245.000000
mean        57.407926
std          7.165219
min          0.040000
25%         53.660745
50%         57.647093
75%         62.012223
max         76.978947
Name: rougher.input.feed_total, dtype: float64


df_golden_recovery_full['rougher.output.concentrate_total'].describe()

count    19416.000000
mean        68.188914
std         12.478522
min          0.000000
25%         66.026048
50%         70.186130
75%         74.362878
max         88.140119
Name: rougher.output.concentrate_total, dtype: float64


df_golden_recovery_full['primary_cleaner.output.concentrate_total'].describe()

count    19069.000000
mean        60.949928
std          9.953770
min          0.000000
25%         59.165854
50%         62.615189
75%         65.725507
max         80.749343
Name: primary_cleaner.output.concentrate_total, dtype: float64


df_golden_recovery_full['final.output.concentrate_total'].describe()

count    19227.000000
mean        68.727562
std          7.423741
min          0.000000
25%         67.985659
50%         69.632597
75%         71.390461
max         80.210644
Name: final.output.concentrate_total, dtype: float64


fig, ax = plt.subplots(2, 2, figsize=(20, 20))
fig.subplots_adjust(hspace=0.3)

# График для Золотоносной руды
x_0 = df_golden_recovery_full['rougher.input.feed_total']
ax[0][0].set_xlabel('Сумма концентраций элементов, %')
ax[0][0].set_ylabel('Количество образцов, шт.')
ax[0][0].set_title('Сумма концентраций элементов в золотоносной руде')
ax[0][0].grid(True)
ax[0][0].hist(x_0, bins=50)

# График для Чернового концентрата
x_1 = df_golden_recovery_full['rougher.output.concentrate_total']
ax[0][1].set_xlabel('Сумма концентраций элементов, %')
ax[0][1].set_ylabel('Количество образцов, шт.')
ax[0][1].set_title('Сумма концентраций элементов в чероновом концентрате')
ax[0][1].grid(True)
ax[0][1].hist(x_1, bins=50)

# График для Концентрата после первой очистки
x_2 = df_golden_recovery_full['primary_cleaner.output.concentrate_total']
ax[1][0].set_xlabel('Сумма концентраций элементов, %')
ax[1][0].set_ylabel('Количество образцов, шт.')
ax[1][0].set_title('Сумма концентраций элементов в концетрате после первой очистки')
ax[1][0].grid(True)
ax[1][0].hist(x_2, bins=50)

# График для Финального концентрата
x_3 = df_golden_recovery_full['final.output.concentrate_total']
ax[1][1].set_xlabel('Сумма концентраций элементов, %')
ax[1][1].set_ylabel('Количество образцов, шт.')
ax[1][1].set_title('Сумма концентраций элементов в финальном концетрате')
ax[1][1].grid(True)
ax[1][1].hist(x_3, bins=50)

plt.show()


bx = df_golden_recovery_full[['rougher.input.feed_total'
                         , 'rougher.output.concentrate_total'
                         , 'primary_cleaner.output.concentrate_total'
                        , 'final.output.concentrate_total']].boxplot(rot=90)
bx.set_xlabel('Этап, Cумма концентраций элементов')
bx.set_ylabel('Концентрация в %')
bx.set_title('Ящик с усами для сумм концентраций элементов на каждом этапе')

plt.show()


display(df_golden_recovery_full[df_golden_recovery_full['rougher.input.feed_total'] <= 10]
 [['rougher.input.feed_au', 'rougher.input.feed_ag', 'rougher.input.feed_pb', 'rougher.input.feed_sol', 'rougher.input.feed_total']]
 .sort_values(by='rougher.input.feed_total')
#  .count()
)


# запишем индекс для аномальных значений
wrong_rougher_input_index = (df_golden_recovery_full[df_golden_recovery_full['rougher.input.feed_total'] <= 10]
 [['rougher.input.feed_au', 'rougher.input.feed_ag', 'rougher.input.feed_pb', 'rougher.input.feed_sol', 'rougher.input.feed_total']]
#  .sort_values(by='rougher.input.feed_total')
 .index
)

# Удаление из full, train и test
df_golden_recovery_full.drop(index=wrong_rougher_input_index, inplace=True, errors='ignore')
df_golden_recovery_train.drop(index=wrong_rougher_input_index, inplace=True, errors='ignore')
df_golden_recovery_test.drop(index=wrong_rougher_input_index, inplace=True, errors='ignore')


display(df_golden_recovery_full[df_golden_recovery_full['rougher.output.concentrate_total'] <= 10]
 [['rougher.output.concentrate_au', 'rougher.output.concentrate_ag', 'rougher.output.concentrate_pb', 'rougher.output.concentrate_sol', 'rougher.output.concentrate_total']]
 .sort_values(by='rougher.output.concentrate_total')
#  .count()
)


# запишем индекс для аномальных значений
wrong_rougher_output_index = (df_golden_recovery_full[df_golden_recovery_full['rougher.output.concentrate_total'] <= 10]
 [['rougher.output.concentrate_au', 'rougher.output.concentrate_ag', 'rougher.output.concentrate_pb', 'rougher.output.concentrate_sol', 'rougher.output.concentrate_total']]
#  .sort_values(by='rougher.output.concentrate_total')
 .index
)

# Удаление из full, train и test
df_golden_recovery_full.drop(index=wrong_rougher_output_index, inplace=True, errors='ignore')
df_golden_recovery_train.drop(index=wrong_rougher_output_index, inplace=True, errors='ignore')
df_golden_recovery_test.drop(index=wrong_rougher_output_index, inplace=True, errors='ignore')


display(df_golden_recovery_full[df_golden_recovery_full['primary_cleaner.output.concentrate_total'] <= 10]
 [['primary_cleaner.output.concentrate_au', 'primary_cleaner.output.concentrate_ag', 'primary_cleaner.output.concentrate_pb', 'primary_cleaner.output.concentrate_sol', 'primary_cleaner.output.concentrate_total']]
 .sort_values(by='primary_cleaner.output.concentrate_total')
#  .count()
)


# запишем индекс для аномальных значений
wrong_primary_cleaner_output_index = (df_golden_recovery_full[df_golden_recovery_full['primary_cleaner.output.concentrate_total'] <= 10]
 [['primary_cleaner.output.concentrate_au', 'primary_cleaner.output.concentrate_ag', 'primary_cleaner.output.concentrate_pb', 'primary_cleaner.output.concentrate_sol', 'primary_cleaner.output.concentrate_total']]
 .index
)

# Удаление из full, train и test
df_golden_recovery_full.drop(index=wrong_primary_cleaner_output_index, inplace=True, errors='ignore')
df_golden_recovery_train.drop(index=wrong_primary_cleaner_output_index, inplace=True, errors='ignore')
df_golden_recovery_test.drop(index=wrong_primary_cleaner_output_index, inplace=True, errors='ignore')


display(df_golden_recovery_full[df_golden_recovery_full['final.output.concentrate_total'] <= 10]
 [['final.output.concentrate_au', 'final.output.concentrate_ag', 'final.output.concentrate_pb', 'final.output.concentrate_sol', 'final.output.concentrate_total']]
 .sort_values(by='final.output.concentrate_total')
#  .count()
)


# запишем индекс для аномальных значений
wrong_final_output_index = (df_golden_recovery_full[df_golden_recovery_full['final.output.concentrate_total'] <= 10]
 [['final.output.concentrate_au', 'final.output.concentrate_ag', 'final.output.concentrate_pb', 'final.output.concentrate_sol', 'final.output.concentrate_total']]
 .index
)

# Удаление из full, train и test
df_golden_recovery_full.drop(index=wrong_final_output_index, inplace=True, errors='ignore')
df_golden_recovery_train.drop(index=wrong_final_output_index, inplace=True, errors='ignore')
df_golden_recovery_test.drop(index=wrong_final_output_index, inplace=True, errors='ignore')


fig, ax = plt.subplots(2, 2, figsize=(20, 20))
fig.subplots_adjust(hspace=0.3)

# График для Золотоносной руды
x_0 = df_golden_recovery_full['rougher.input.feed_total']
ax[0][0].set_xlabel('Сумма концентраций элементов, %')
ax[0][0].set_ylabel('Количество образцов, шт.')
ax[0][0].set_title('Сумма концентраций элементов в золотоносной руде')
ax[0][0].grid(True)
ax[0][0].hist(x_0, bins=50)

# График для Чернового концентрата
x_1 = df_golden_recovery_full['rougher.output.concentrate_total']
ax[0][1].set_xlabel('Сумма концентраций элементов, %')
ax[0][1].set_ylabel('Количество образцов, шт.')
ax[0][1].set_title('Сумма концентраций элементов в чероновом концентрате')
ax[0][1].grid(True)
ax[0][1].hist(x_1, bins=50)

# График для Концентрата после первой очистки
x_2 = df_golden_recovery_full['primary_cleaner.output.concentrate_total']
ax[1][0].set_xlabel('Сумма концентраций элементов, %')
ax[1][0].set_ylabel('Количество образцов, шт.')
ax[1][0].set_title('Сумма концентраций элементов в концетрате после первой очистки')
ax[1][0].grid(True)
ax[1][0].hist(x_2, bins=50)

# График для Финального концентрата
x_3 = df_golden_recovery_full['final.output.concentrate_total']
ax[1][1].set_xlabel('Сумма концентраций элементов, %')
ax[1][1].set_ylabel('Количество образцов, шт.')
ax[1][1].set_title('Сумма концентраций элементов в финальном концетрате')
ax[1][1].grid(True)
ax[1][1].hist(x_3, bins=50)

plt.show()


bx = df_golden_recovery_full[['rougher.input.feed_total'
                         , 'rougher.output.concentrate_total'
                         , 'primary_cleaner.output.concentrate_total'
                        , 'final.output.concentrate_total']].boxplot(rot=90)
bx.set_xlabel('Этап, Cумма концентраций элементов')
bx.set_ylabel('Концентрация в %')
bx.set_title('Ящик с усами для сумм концентраций элементов на каждом этапе')

plt.show()


df_golden_recovery_test.columns[df_golden_recovery_test.columns.str.contains('size')]

Index(['primary_cleaner.input.feed_size', 'rougher.input.feed_size'], dtype='object')


df_golden_recovery_train.columns[df_golden_recovery_train.columns.str.contains('size')]

Index(['primary_cleaner.input.feed_size', 'rougher.input.feed_size'], dtype='object')


fig, ax = plt.subplots(2, 2, figsize=(18, 18))
fig.subplots_adjust(hspace=0.3)

# График kde для train
x_0 = df_golden_recovery_train['rougher.input.feed_size']
ax[0][0].set_xlabel('Размер гранул')
ax[0][0].set_ylabel('Плотность распределения')
ax[0][0].set_title('Плотность распределения размера гранул перед флотацией, train')
ax[0][0].grid(True)
sns.kdeplot(x_0, ax=ax[0][0], shade=True, color='blue')
ax[0][0].set_xlim(-10, 510)

# График boxplot для train
x_1 = df_golden_recovery_train['rougher.input.feed_size']
# ax[0][1].set_ylabel('Этап очистки, параметр')
ax[0][1].set_xlabel('Размер гранул')
ax[0][1].set_title('Ящик с усами для размера гранул перед флотацией, train')
ax[0][1].grid(True)
sns.boxplot(x_1, ax=ax[0][1], color='blue')
ax[0][1].set_xlim(-10, 510)

# График kde для test
x_2 = df_golden_recovery_test['rougher.input.feed_size']
ax[1][0].set_xlabel('Размер гранул')
ax[1][0].set_ylabel('Плотность распределения')
ax[1][0].set_title('Плотность распределения размера гранул перед флотацией, test')
ax[1][0].grid(True)
sns.kdeplot(x_2, ax=ax[1][0], shade=True, color='red')
ax[1][0].set_xlim(-10, 510)

# График boxplot для test
x_3 = df_golden_recovery_test['rougher.input.feed_size']
# ax[1][1].set_ylabel('Этап очистки, параметр')
ax[1][1].set_xlabel('Размер гранул')
ax[1][1].set_title('Ящик с усами для размера гранул перед флотацией, test')
ax[1][1].grid(True)
sns.boxplot(x_3, ax=ax[1][1], color='red')
ax[1][1].set_xlim(-10, 510)

plt.show()


df_golden_recovery_train['rougher.input.feed_size'].describe()

count    13433.000000
mean        60.084403
std         22.582113
min          9.659576
25%         48.986490
50%         55.344437
75%         65.860791
max        484.967466
Name: rougher.input.feed_size, dtype: float64


df_golden_recovery_test['rougher.input.feed_size'].describe()

count    5110.000000
mean       56.004693
std        19.170091
min         0.046369
25%        43.893832
50%        50.974866
75%        62.405665
max       392.494040
Name: rougher.input.feed_size, dtype: float64


# эта функция расчитывает доверительный интревал( 2 границы) для указанной квартили и количества циклов
def confidential_interval(data, data_quantile, lower_interval_quantile, upper_interval_quantile, bootstrap_samples):
    # зафиксируем условно случайные состояния для выборок
    state = RandomState(54321)
    # сохраните значения 97%-квантилей в переменной values
    values = []
    # получаем выборки, по каждой рачитываем нужный параметр и накапливаем данные
    for i in range(bootstrap_samples):
        subsample = data.sample(frac=1, replace=True, random_state=state)
        # накапливаем данные 
        values.append(subsample.quantile(q=data_quantile))

    values = pd.Series(values)
  
    # выводим на экран доверительные интервал
    display('({:.2f},{:.2f})'.format(values.quantile(lower_interval_quantile), values.quantile(upper_interval_quantile)))


confidential_interval(df_golden_recovery_train['rougher.input.feed_size'], 0.97, 0.025, 0.975, 1000)

'(90.04,93.58)'


confidential_interval(df_golden_recovery_test['rougher.input.feed_size'], 0.97, 0.025, 0.975, 1000)

'(89.92,94.15)'


confidential_interval(df_golden_recovery_full['rougher.input.feed_size'], 0.97, 0.025, 0.975, 1000)

'(90.61,93.37)'


fig, ax = plt.subplots(2, 2, figsize=(18, 18))
fig.subplots_adjust(hspace=0.3)

# График kde для train
x_0 = df_golden_recovery_train['primary_cleaner.input.feed_size']
ax[0][0].set_xlabel('Размер гранул')
ax[0][0].set_ylabel('Плотность распределения')
ax[0][0].set_title('Плотность распределения размера гранул после флотации, train')
ax[0][0].grid(True)
sns.kdeplot(x_0, ax=ax[0][0], shade=True, color='blue')
ax[0][0].set_xlim(-1, 17)

# График boxplot для train
x_1 = df_golden_recovery_train['primary_cleaner.input.feed_size']
# ax[0][1].set_ylabel('Этап очистки, параметр')
ax[0][1].set_xlabel('Размер гранул')
ax[0][1].set_title('Ящик с усами для размера гранул после флотации, train')
ax[0][1].grid(True)
sns.boxplot(x_1, ax=ax[0][1], color='blue')
ax[0][1].set_xlim(-1, 17)

# График kde для test
x_2 = df_golden_recovery_test['primary_cleaner.input.feed_size']
ax[1][0].set_xlabel('Размер гранул')
ax[1][0].set_ylabel('Плотность распределения')
ax[1][0].set_title('Плотность распределения размера гранул после флотации, test')
ax[1][0].grid(True)
sns.kdeplot(x_2, ax=ax[1][0], shade=True, color='red')
ax[1][0].set_xlim(-1, 17)

# График boxplot для test
x_3 = df_golden_recovery_test['primary_cleaner.input.feed_size']
# ax[1][1].set_ylabel('Этап очистки, параметр')
ax[1][1].set_xlabel('Размер гранул')
ax[1][1].set_title('Ящик с усами для размера гранул после флотации, test')
ax[1][1].grid(True)
sns.boxplot(x_3, ax=ax[1][1], color='red')
ax[1][1].set_xlim(-1, 17)

plt.show()


df_golden_recovery_train['primary_cleaner.input.feed_size'].describe()

count    13571.000000
mean         7.329046
std          0.615204
min          1.080000
25%          6.970000
50%          7.300000
75%          7.700000
max         10.470000
Name: primary_cleaner.input.feed_size, dtype: float64


df_golden_recovery_test['primary_cleaner.input.feed_size'].describe()

count    5111.000000
mean        7.258758
std         0.604802
min         5.650000
25%         6.890000
50%         7.250000
75%         7.590000
max        15.500000
Name: primary_cleaner.input.feed_size, dtype: float64


confidential_interval(df_golden_recovery_train['primary_cleaner.input.feed_size'], 0.97, 0.025, 0.975, 1000)

'(8.48,8.51)'


confidential_interval(df_golden_recovery_test['primary_cleaner.input.feed_size'], 0.97, 0.025, 0.975, 1000)

'(8.37,8.47)'


confidential_interval(df_golden_recovery_full['primary_cleaner.input.feed_size'], 0.97, 0.025, 0.975, 1000)

'(8.47,8.50)'


rs = 54321


columns_in_test_for_rougher_recovery = df_golden_recovery_test.columns[df_golden_recovery_test.columns.str.contains('rougher')].to_list()
columns_in_test_for_rougher_recovery

['rougher.input.feed_ag',
 'rougher.input.feed_pb',
 'rougher.input.feed_rate',
 'rougher.input.feed_size',
 'rougher.input.feed_sol',
 'rougher.input.feed_au',
 'rougher.input.floatbank10_sulfate',
 'rougher.input.floatbank10_xanthate',
 'rougher.input.floatbank11_sulfate',
 'rougher.input.floatbank11_xanthate',
 'rougher.state.floatbank10_a_air',
 'rougher.state.floatbank10_a_level',
 'rougher.state.floatbank10_b_air',
 'rougher.state.floatbank10_b_level',
 'rougher.state.floatbank10_c_air',
 'rougher.state.floatbank10_c_level',
 'rougher.state.floatbank10_d_air',
 'rougher.state.floatbank10_d_level',
 'rougher.state.floatbank10_e_air',
 'rougher.state.floatbank10_e_level',
 'rougher.state.floatbank10_f_air',
 'rougher.state.floatbank10_f_level']


columns_in_test

['primary_cleaner.input.sulfate',
 'primary_cleaner.input.depressant',
 'primary_cleaner.input.feed_size',
 'primary_cleaner.input.xanthate',
 'primary_cleaner.state.floatbank8_a_air',
 'primary_cleaner.state.floatbank8_a_level',
 'primary_cleaner.state.floatbank8_b_air',
 'primary_cleaner.state.floatbank8_b_level',
 'primary_cleaner.state.floatbank8_c_air',
 'primary_cleaner.state.floatbank8_c_level',
 'primary_cleaner.state.floatbank8_d_air',
 'primary_cleaner.state.floatbank8_d_level',
 'rougher.input.feed_ag',
 'rougher.input.feed_pb',
 'rougher.input.feed_rate',
 'rougher.input.feed_size',
 'rougher.input.feed_sol',
 'rougher.input.feed_au',
 'rougher.input.floatbank10_sulfate',
 'rougher.input.floatbank10_xanthate',
 'rougher.input.floatbank11_sulfate',
 'rougher.input.floatbank11_xanthate',
 'rougher.state.floatbank10_a_air',
 'rougher.state.floatbank10_a_level',
 'rougher.state.floatbank10_b_air',
 'rougher.state.floatbank10_b_level',
 'rougher.state.floatbank10_c_air',
 'rougher.state.floatbank10_c_level',
 'rougher.state.floatbank10_d_air',
 'rougher.state.floatbank10_d_level',
 'rougher.state.floatbank10_e_air',
 'rougher.state.floatbank10_e_level',
 'rougher.state.floatbank10_f_air',
 'rougher.state.floatbank10_f_level',
 'secondary_cleaner.state.floatbank2_a_air',
 'secondary_cleaner.state.floatbank2_a_level',
 'secondary_cleaner.state.floatbank2_b_air',
 'secondary_cleaner.state.floatbank2_b_level',
 'secondary_cleaner.state.floatbank3_a_air',
 'secondary_cleaner.state.floatbank3_a_level',
 'secondary_cleaner.state.floatbank3_b_air',
 'secondary_cleaner.state.floatbank3_b_level',
 'secondary_cleaner.state.floatbank4_a_air',
 'secondary_cleaner.state.floatbank4_a_level',
 'secondary_cleaner.state.floatbank4_b_air',
 'secondary_cleaner.state.floatbank4_b_level',
 'secondary_cleaner.state.floatbank5_a_air',
 'secondary_cleaner.state.floatbank5_a_level',
 'secondary_cleaner.state.floatbank5_b_air',
 'secondary_cleaner.state.floatbank5_b_level',
 'secondary_cleaner.state.floatbank6_a_air',
 'secondary_cleaner.state.floatbank6_a_level']


features_train_rougher_recovery = df_golden_recovery_train[columns_in_test_for_rougher_recovery]
features_train_rougher_recovery.shape

(13571, 22)


target_train_rougher_recovery = df_golden_recovery_train['rougher.output.recovery']
target_train_rougher_recovery.shape
# type(target_train_rougher_recovery)

(13571,)


features_test_rougher_recovery = df_golden_recovery_test[columns_in_test_for_rougher_recovery]
features_test_rougher_recovery.shape

(5111, 22)


target_test_rougher_recovery = df_golden_recovery_full.loc[df_golden_recovery_test.index, 'rougher.output.recovery']
target_test_rougher_recovery.shape

(5111,)


features_train_final_recovery = df_golden_recovery_train[columns_in_test]
features_train_final_recovery.shape

(13571, 52)


target_train_final_recovery = df_golden_recovery_train['final.output.recovery']
target_train_final_recovery.shape

(13571,)


features_test_final_recovery = df_golden_recovery_test
features_test_final_recovery.shape

(5111, 52)


target_test_final_recovery = df_golden_recovery_full.loc[df_golden_recovery_test.index, 'final.output.recovery']
target_test_final_recovery.shape

(5111,)


cv_folders = KFold(n_splits=5
                   , shuffle=True
                   , random_state=rs
                  )


def smape_score_function(target, prediction):
    smape = sum(abs(target - prediction) * 2 / (abs(target) + abs(prediction))) / len(target) * 100
    return smape


def total_smape_score_function(target_rougher, prediction_rougher, target_final, prediction_final):
    total_smape = (0.25 * smape_score_function(prediction_rougher, target_rougher)
                   + 0.75 * smape_score_function(prediction_final, target_final))
    return total_smape


smape_scorer = make_scorer(smape_score_function, greater_is_better=False)
smape_scorer

make_scorer(smape_score_function, greater_is_better=False)


dummy_model = DummyRegressor(strategy='median')
dummy_model.fit(features_train_rougher_recovery, target_train_rougher_recovery)
dummy_prediction_rougher_train = dummy_model.predict(features_train_rougher_recovery)
smape_score_function(target_train_rougher_recovery, dummy_prediction_rougher_train)

7.20824668664892


dummy_model = DummyRegressor(strategy='median')
dummy_model.fit(features_train_rougher_recovery, target_train_rougher_recovery) # обучение на train
dummy_prediction_rougher = dummy_model.predict(features_test_rougher_recovery) # предсказание на test
smape_score_function(target_test_rougher_recovery, dummy_prediction_rougher) # оценка на test

4.7691855079609216


dummy_model = DummyRegressor(strategy='median')
dummy_model.fit(features_train_final_recovery, target_train_final_recovery)
dummy_prediction_final_train = dummy_model.predict(features_train_final_recovery)
smape_score_function(target_train_final_recovery, dummy_prediction_final_train)

10.041347277039886


dummy_model = DummyRegressor(strategy='median')
dummy_model.fit(features_train_final_recovery, target_train_final_recovery)
dummy_prediction_final = dummy_model.predict(features_test_final_recovery)
smape_score_function(target_test_final_recovery, dummy_prediction_final)

8.189838531672008


total_smape_score_function(target_train_rougher_recovery, dummy_prediction_rougher_train
                           , target_train_final_recovery, dummy_prediction_final_train)

9.333072129442144


total_smape_score_function(target_test_rougher_recovery, dummy_prediction_rougher
                           , target_test_final_recovery, dummy_prediction_final)

7.334675275744235


# обработка численных признаков специально для алгоритмов с деревьями: заполнение пропусков аномальными значениями,
# стандартизация и маштабирование
preprocessor_steps_tree = [('imputer', SimpleImputer(missing_values=np.nan, strategy='constant', fill_value=-1000))]
preprocessor_tree = Pipeline(preprocessor_steps_tree)


# обработка численных признаков специально для линейных моделей: заполнение пропусков нулями + метки 
# о заполнении пропуска в отдельном столбце,
# стандартизация и маштабирование
preprocessor_steps_linear = [('imputer', SimpleImputer(missing_values=np.nan
                                                       , strategy='constant'
                                                       , fill_value=0
                                                       , add_indicator=True
                                                      )
                             )
             , ('scaler', StandardScaler())]
preprocessor_linear = Pipeline(preprocessor_steps_linear)


# шаги для pipeline в GridSearchCV для моделей с деревями
steps_tree = [('preprocessor', preprocessor_tree), ('regressor', DecisionTreeRegressor(random_state=rs))]


# шаги для pipeline в GridSearchCV для линейных моделей
steps_linear = [('preprocessor', preprocessor_linear), ('regressor', LinearRegression(n_jobs=-1))]


# параметры для улучшения моделей в GridSearchCV для моделей с деревьями

params_tree = [{'regressor': [DecisionTreeRegressor(random_state=rs)]
             , 'regressor__max_depth': [2, 3, 4, 5, 6, 7, 8, 9, 10]
             , 'regressor__max_features': ['sqrt'] #default = 1  
          } 
          , {'regressor': [RandomForestRegressor(random_state=rs)]
            , 'regressor__bootstrap': [False]  #default = True
             , 'regressor__max_depth': [2, 3, 4, 5, 6, 7, 8, 9, 10]
             , 'regressor__max_features': ['sqrt'] #default = 1
             , 'regressor__n_estimators': [2, 3, 4, 5, 6, 7, 8, 9, 10] # default = 100
          }          
        ]


# параметры для улучшения моделей в GridSearchCV для линейных алгоритмов - регулиризация

params_linear = [{'regressor': [LinearRegression()]
                } 
          , {'regressor': [Ridge()]
            , 'regressor__alpha': [0.2, 0.4, 0.6, 0.8, 1] # default 1.0
             , 'regressor__solver': ['auto', 'svd', 'cholesky', 'lsqr', 'sparse_cg', 'sag', 'saga']
             , 'regressor__random_state': [None, rs]
          } 
          , {'regressor': [Lasso()]
            , 'regressor__alpha': [0.2, 0.4, 0.6, 0.8, 1] # default 1.0
             , 'regressor__random_state': [None, rs]
          } 
        ]


#на вход идут признаки, целевой признак, шаги по умолчанию для pipeline, метод расчета (smape_scorer), 
# параметр для отображения процесса работы
# на выход лучший обученный объект GridSearchCV
#
def pipeline_gridsearchcv(X, y, steps, params, scoring, verbose = 1):
    
    # формируем итоговый pipeline 
    pipe = Pipeline(steps)
 
    # собираем все вместе, запускаем поиск
    grid = GridSearchCV(pipe, param_grid=params
                        , cv=cv_folders
                        , n_jobs=-1
                        , scoring=scoring
                        , verbose=verbose
                        , error_score='raise')
    grid.fit(X, y)
    
    #Посмотрим наилучшие подобранные параметры
    display('')
    display('Параметры лучшей модели:', grid.best_params_)
    display('Значение лучшей метрики качества:', -grid.best_score_)
    
    return grid


# создадим pipeline, подберем параметры для лучшей модели на кросс-валидации
grid_search_rougher_recovery_linear = pipeline_gridsearchcv(features_train_rougher_recovery
                                                   , target_train_rougher_recovery
                                                   , steps_linear
                                                   , params_linear
                                                   , smape_scorer
                                                   , verbose = 1)
grid_search_rougher_recovery_linear.best_estimator_

Fitting 5 folds for each of 81 candidates, totalling 405 fits

''

'Параметры лучшей модели:'

{'regressor': Ridge(alpha=0.6, solver='sag'),
 'regressor__alpha': 0.6,
 'regressor__random_state': None,
 'regressor__solver': 'sag'}

'Значение лучшей метрики качества:'

5.06973235931769

Pipeline(steps=[('preprocessor',
                 Pipeline(steps=[('imputer',
                                  SimpleImputer(add_indicator=True,
                                                fill_value=0,
                                                strategy='constant')),
                                 ('scaler', StandardScaler())])),
                ('regressor', Ridge(alpha=0.6, solver='sag'))])


(pd.DataFrame(grid_search_rougher_recovery_linear.cv_results_)
  [['params', 'mean_test_score', 'rank_test_score', 'mean_fit_time', 'mean_score_time']]
  .sort_values(by='mean_test_score', ascending=False)
  .head(50))


# создадим pipeline, подберем параметры для лучшей модели на кросс-валидации
grid_search_rougher_recovery_tree = pipeline_gridsearchcv(features_train_rougher_recovery
                                                   , target_train_rougher_recovery
                                                   , steps_tree
                                                   , params_tree
                                                   , smape_scorer
                                                   , verbose = 1)
grid_search_rougher_recovery_tree.best_estimator_

Fitting 5 folds for each of 90 candidates, totalling 450 fits

''

'Параметры лучшей модели:'

{'regressor': RandomForestRegressor(bootstrap=False, max_depth=10, max_features='sqrt',
                       n_estimators=10, random_state=54321),
 'regressor__bootstrap': False,
 'regressor__max_depth': 10,
 'regressor__max_features': 'sqrt',
 'regressor__n_estimators': 10}

'Значение лучшей метрики качества:'

3.3645503254743807

Pipeline(steps=[('preprocessor',
                 Pipeline(steps=[('imputer',
                                  SimpleImputer(fill_value=-1000,
                                                strategy='constant'))])),
                ('regressor',
                 RandomForestRegressor(bootstrap=False, max_depth=10,
                                       max_features='sqrt', n_estimators=10,
                                       random_state=54321))])


(pd.DataFrame(grid_search_rougher_recovery_tree.cv_results_)
  [['params', 'mean_test_score', 'rank_test_score', 'mean_fit_time', 'mean_score_time']]
  .sort_values(by='mean_test_score', ascending=False)
  .head(50))


if abs(grid_search_rougher_recovery_tree.best_score_) < abs(grid_search_rougher_recovery_linear.best_score_):
    best_model_rougher_recovery = grid_search_rougher_recovery_tree.best_estimator_
else:
    best_model_rougher_recovery = grid_search_rougher_recovery_linear.best_estimator_
    
best_model_rougher_recovery

Pipeline(steps=[('preprocessor',
                 Pipeline(steps=[('imputer',
                                  SimpleImputer(fill_value=-1000,
                                                strategy='constant'))])),
                ('regressor',
                 RandomForestRegressor(bootstrap=False, max_depth=10,
                                       max_features='sqrt', n_estimators=10,
                                       random_state=54321))])


best_model_rougher_recovery

Pipeline(steps=[('preprocessor',
                 Pipeline(steps=[('imputer',
                                  SimpleImputer(fill_value=-1000,
                                                strategy='constant'))])),
                ('regressor',
                 RandomForestRegressor(bootstrap=False, max_depth=10,
                                       max_features='sqrt', n_estimators=10,
                                       random_state=54321))])


# создадим pipeline, подберем параметры для лучшей модели на кросс-валидации
grid_search_final_recovery_linear = pipeline_gridsearchcv(features_train_final_recovery
                                                   , target_train_final_recovery
                                                   , steps_linear
                                                   , params_linear
                                                   , smape_scorer
                                                   , verbose = 1)
grid_search_final_recovery_linear.best_estimator_

Fitting 5 folds for each of 81 candidates, totalling 405 fits

''

'Параметры лучшей модели:'

{'regressor': Ridge(alpha=1, solver='lsqr'),
 'regressor__alpha': 1,
 'regressor__random_state': None,
 'regressor__solver': 'lsqr'}

'Значение лучшей метрики качества:'

7.940929036270849

Pipeline(steps=[('preprocessor',
                 Pipeline(steps=[('imputer',
                                  SimpleImputer(add_indicator=True,
                                                fill_value=0,
                                                strategy='constant')),
                                 ('scaler', StandardScaler())])),
                ('regressor', Ridge(alpha=1, solver='lsqr'))])


(pd.DataFrame(grid_search_final_recovery_linear.cv_results_)
  [['params', 'mean_test_score', 'rank_test_score', 'mean_fit_time', 'mean_score_time']]
  .sort_values(by='mean_test_score', ascending=False)
  .head(50))


# создадим pipeline, подберем параметры для лучшей модели на кросс-валидации
grid_search_final_recovery_tree = pipeline_gridsearchcv(features_train_final_recovery
                                                   , target_train_final_recovery
                                                   , steps_tree
                                                   , params_tree
                                                   , smape_scorer
                                                   , verbose = 1)
grid_search_final_recovery_tree.best_estimator_

Fitting 5 folds for each of 90 candidates, totalling 450 fits

''

'Параметры лучшей модели:'

{'regressor': RandomForestRegressor(bootstrap=False, max_depth=10, max_features='sqrt',
                       n_estimators=9, random_state=54321),
 'regressor__bootstrap': False,
 'regressor__max_depth': 10,
 'regressor__max_features': 'sqrt',
 'regressor__n_estimators': 9}

'Значение лучшей метрики качества:'

6.410720756958277

Pipeline(steps=[('preprocessor',
                 Pipeline(steps=[('imputer',
                                  SimpleImputer(fill_value=-1000,
                                                strategy='constant'))])),
                ('regressor',
                 RandomForestRegressor(bootstrap=False, max_depth=10,
                                       max_features='sqrt', n_estimators=9,
                                       random_state=54321))])


(pd.DataFrame(grid_search_final_recovery_tree.cv_results_)
  [['params', 'mean_test_score', 'rank_test_score', 'mean_fit_time', 'mean_score_time']]
  .sort_values(by='mean_test_score', ascending=False)
  .head(50))


if abs(grid_search_final_recovery_tree.best_score_) < abs(grid_search_final_recovery_linear.best_score_):
    best_model_final_recovery = grid_search_final_recovery_tree.best_estimator_
else:
    best_model_final_recovery = grid_search_final_recovery_linear.best_estimator_
    
best_model_final_recovery

Pipeline(steps=[('preprocessor',
                 Pipeline(steps=[('imputer',
                                  SimpleImputer(fill_value=-1000,
                                                strategy='constant'))])),
                ('regressor',
                 RandomForestRegressor(bootstrap=False, max_depth=10,
                                       max_features='sqrt', n_estimators=9,
                                       random_state=54321))])


best_model_final_recovery

Pipeline(steps=[('preprocessor',
                 Pipeline(steps=[('imputer',
                                  SimpleImputer(fill_value=-1000,
                                                strategy='constant'))])),
                ('regressor',
                 RandomForestRegressor(bootstrap=False, max_depth=10,
                                       max_features='sqrt', n_estimators=9,
                                       random_state=54321))])


#расчитаем sMAPE для флотации на тестовой выборке
# prediction_rougher_recovery = grid_search_rougher_recovery.predict(features_test_rougher_recovery)
prediction_rougher_recovery = best_model_rougher_recovery.predict(features_test_rougher_recovery)
smape_score_function(target_test_rougher_recovery, prediction_rougher_recovery)

4.154530043747265


#расчитаем sMAPE для флотации на тестовой выборке
# prediction_final_recovery = grid_search_final_recovery.predict(features_test_final_recovery)
prediction_final_recovery = best_model_final_recovery.predict(features_test_final_recovery)
smape_score_function(target_test_final_recovery, prediction_final_recovery)

7.990810432021004


# расчитаем Итоговую sMAPE на тестовой выборке
total_smape_score_function(target_test_rougher_recovery, prediction_rougher_recovery
                          , target_test_final_recovery, prediction_final_recovery)

7.031740334952569


best_model_rougher_recovery

Pipeline(steps=[('preprocessor',
                 Pipeline(steps=[('imputer',
                                  SimpleImputer(fill_value=-1000,
                                                strategy='constant'))])),
                ('regressor',
                 RandomForestRegressor(bootstrap=False, max_depth=10,
                                       max_features='sqrt', n_estimators=10,
                                       random_state=54321))])


best_model_final_recovery

Pipeline(steps=[('preprocessor',
                 Pipeline(steps=[('imputer',
                                  SimpleImputer(fill_value=-1000,
                                                strategy='constant'))])),
                ('regressor',
                 RandomForestRegressor(bootstrap=False, max_depth=10,
                                       max_features='sqrt', n_estimators=9,
                                       random_state=54321))])

	final.output.concentrate_ag	final.output.concentrate_pb	final.output.concentrate_sol	final.output.concentrate_au	final.output.recovery	final.output.tail_ag	final.output.tail_pb	final.output.tail_sol	final.output.tail_au	primary_cleaner.input.sulfate	...	secondary_cleaner.state.floatbank4_a_air	secondary_cleaner.state.floatbank4_a_level	secondary_cleaner.state.floatbank4_b_air	secondary_cleaner.state.floatbank4_b_level	secondary_cleaner.state.floatbank5_a_air	secondary_cleaner.state.floatbank5_a_level	secondary_cleaner.state.floatbank5_b_air	secondary_cleaner.state.floatbank5_b_level	secondary_cleaner.state.floatbank6_a_air	secondary_cleaner.state.floatbank6_a_level
date
2016-01-15 00:00:00	6.055403	9.889648	5.507324	42.192020	70.541216	10.411962	0.895447	16.904297	2.143149	127.092003	...	14.016835	-502.488007	12.099931	-504.715942	9.925633	-498.310211	8.079666	-500.470978	14.151341	-605.841980
2016-01-15 01:00:00	6.029369	9.968944	5.257781	42.701629	69.266198	10.462676	0.927452	16.634514	2.224930	125.629232	...	13.992281	-505.503262	11.950531	-501.331529	10.039245	-500.169983	7.984757	-500.582168	13.998353	-599.787184
2016-01-15 02:00:00	6.055926	10.213995	5.383759	42.657501	68.116445	10.507046	0.953716	16.208849	2.257889	123.819808	...	14.015015	-502.520901	11.912783	-501.133383	10.070913	-500.129135	8.013877	-500.517572	14.028663	-601.427363
2016-01-15 03:00:00	6.047977	9.977019	4.858634	42.689819	68.347543	10.422762	0.883763	16.532835	2.146849	122.270188	...	14.036510	-500.857308	11.999550	-501.193686	9.970366	-499.201640	7.977324	-500.255908	14.005551	-599.996129
2016-01-15 04:00:00	6.148599	10.142511	4.939416	42.774141	66.927016	10.360302	0.792826	16.525686	2.055292	117.988169	...	14.027298	-499.838632	11.953070	-501.053894	9.925709	-501.686727	7.894242	-500.356035	13.996647	-601.496691

	primary_cleaner.input.sulfate	primary_cleaner.input.depressant	primary_cleaner.input.feed_size	primary_cleaner.input.xanthate	primary_cleaner.state.floatbank8_a_air	primary_cleaner.state.floatbank8_a_level	primary_cleaner.state.floatbank8_b_air	primary_cleaner.state.floatbank8_b_level	primary_cleaner.state.floatbank8_c_air	primary_cleaner.state.floatbank8_c_level	...	secondary_cleaner.state.floatbank4_a_air	secondary_cleaner.state.floatbank4_a_level	secondary_cleaner.state.floatbank4_b_air	secondary_cleaner.state.floatbank4_b_level	secondary_cleaner.state.floatbank5_a_air	secondary_cleaner.state.floatbank5_a_level	secondary_cleaner.state.floatbank5_b_air	secondary_cleaner.state.floatbank5_b_level	secondary_cleaner.state.floatbank6_a_air	secondary_cleaner.state.floatbank6_a_level
date
2016-09-01 00:59:59	210.800909	14.993118	8.080000	1.005021	1398.981301	-500.225577	1399.144926	-499.919735	1400.102998	-500.704369	...	12.023554	-497.795834	8.016656	-501.289139	7.946562	-432.317850	4.872511	-500.037437	26.705889	-499.709414
2016-09-01 01:59:59	215.392455	14.987471	8.080000	0.990469	1398.777912	-500.057435	1398.055362	-499.778182	1396.151033	-499.240168	...	12.058140	-498.695773	8.130979	-499.634209	7.958270	-525.839648	4.878850	-500.162375	25.019940	-499.819438
2016-09-01 02:59:59	215.259946	12.884934	7.786667	0.996043	1398.493666	-500.868360	1398.860436	-499.764529	1398.075709	-502.151509	...	11.962366	-498.767484	8.096893	-500.827423	8.071056	-500.801673	4.905125	-499.828510	24.994862	-500.622559
2016-09-01 03:59:59	215.336236	12.006805	7.640000	0.863514	1399.618111	-498.863574	1397.440120	-499.211024	1400.129303	-498.355873	...	12.033091	-498.350935	8.074946	-499.474407	7.897085	-500.868509	4.931400	-499.963623	24.948919	-498.709987
2016-09-01 04:59:59	199.099327	10.682530	7.530000	0.805575	1401.268123	-500.808305	1398.128818	-499.504543	1402.172226	-500.810606	...	12.025367	-500.786497	8.054678	-500.397500	8.107890	-509.526725	4.957674	-500.360026	25.003331	-500.856333

	final.output.concentrate_ag	final.output.concentrate_pb	final.output.concentrate_sol	final.output.concentrate_au	final.output.recovery	final.output.tail_ag	final.output.tail_pb	final.output.tail_sol	final.output.tail_au	primary_cleaner.input.sulfate	...	secondary_cleaner.state.floatbank4_a_air	secondary_cleaner.state.floatbank4_a_level	secondary_cleaner.state.floatbank4_b_air	secondary_cleaner.state.floatbank4_b_level	secondary_cleaner.state.floatbank5_a_air	secondary_cleaner.state.floatbank5_a_level	secondary_cleaner.state.floatbank5_b_air	secondary_cleaner.state.floatbank5_b_level	secondary_cleaner.state.floatbank6_a_air	secondary_cleaner.state.floatbank6_a_level
date
2016-01-15 00:00:00	6.055403	9.889648	5.507324	42.192020	70.541216	10.411962	0.895447	16.904297	2.143149	127.092003	...	14.016835	-502.488007	12.099931	-504.715942	9.925633	-498.310211	8.079666	-500.470978	14.151341	-605.841980
2016-01-15 01:00:00	6.029369	9.968944	5.257781	42.701629	69.266198	10.462676	0.927452	16.634514	2.224930	125.629232	...	13.992281	-505.503262	11.950531	-501.331529	10.039245	-500.169983	7.984757	-500.582168	13.998353	-599.787184
2016-01-15 02:00:00	6.055926	10.213995	5.383759	42.657501	68.116445	10.507046	0.953716	16.208849	2.257889	123.819808	...	14.015015	-502.520901	11.912783	-501.133383	10.070913	-500.129135	8.013877	-500.517572	14.028663	-601.427363
2016-01-15 03:00:00	6.047977	9.977019	4.858634	42.689819	68.347543	10.422762	0.883763	16.532835	2.146849	122.270188	...	14.036510	-500.857308	11.999550	-501.193686	9.970366	-499.201640	7.977324	-500.255908	14.005551	-599.996129
2016-01-15 04:00:00	6.148599	10.142511	4.939416	42.774141	66.927016	10.360302	0.792826	16.525686	2.055292	117.988169	...	14.027298	-499.838632	11.953070	-501.053894	9.925709	-501.686727	7.894242	-500.356035	13.996647	-601.496691

	rougher.input.feed_au	rougher.output.concentrate_au	primary_cleaner.output.concentrate_au	final.output.concentrate_au	rougher.input.feed_ag	rougher.output.concentrate_ag	primary_cleaner.output.concentrate_ag	final.output.concentrate_ag	rougher.input.feed_pb	rougher.output.concentrate_pb	primary_cleaner.output.concentrate_pb	final.output.concentrate_pb	rougher.input.feed_sol	rougher.output.concentrate_sol	primary_cleaner.output.concentrate_sol	final.output.concentrate_sol
rougher.input.feed_au	1.000000	0.232677	-0.119264	0.035095	0.845274	0.305916	0.442915	0.315817	0.687003	0.004302	0.204538	-0.079812	0.094392	0.221388	0.476985	0.474520
rougher.output.concentrate_au	0.232677	1.000000	0.134588	0.078927	0.136412	0.619910	0.031454	0.044347	0.071388	0.495720	-0.045795	-0.030951	-0.090094	0.868293	0.011100	0.225260
primary_cleaner.output.concentrate_au	-0.119264	0.134588	1.000000	0.239687	-0.183847	-0.139869	0.131055	-0.200367	-0.226356	-0.053267	0.306172	0.010703	-0.091333	0.061587	0.165738	-0.054210
final.output.concentrate_au	0.035095	0.078927	0.239687	1.000000	0.020728	0.038816	-0.040442	-0.038373	0.011408	-0.006444	0.129691	0.374400	0.219287	0.083825	-0.006042	0.261225
rougher.input.feed_ag	0.845274	0.136412	-0.183847	0.020728	1.000000	0.500300	0.547928	0.456279	0.496579	-0.131754	0.032741	-0.207999	0.115878	0.137356	0.553543	0.479259
rougher.output.concentrate_ag	0.305916	0.619910	-0.139869	0.038816	0.500300	1.000000	0.359826	0.315558	0.082539	0.295688	-0.051920	-0.253089	0.159997	0.685685	0.301270	0.359009
primary_cleaner.output.concentrate_ag	0.442915	0.031454	0.131055	-0.040442	0.547928	0.359826	1.000000	0.509877	0.176723	-0.092061	0.368670	-0.036563	0.232016	0.049560	0.724903	0.362700
final.output.concentrate_ag	0.315817	0.044347	-0.200367	-0.038373	0.456279	0.315558	0.509877	1.000000	0.074866	-0.127291	-0.152814	0.063570	-0.036783	-0.018497	0.332771	0.380142
rougher.input.feed_pb	0.687003	0.071388	-0.226356	0.011408	0.496579	0.082539	0.176723	0.074866	1.000000	0.457030	0.458708	0.301446	0.090636	0.191396	0.185224	0.185576
rougher.output.concentrate_pb	0.004302	0.495720	-0.053267	-0.006444	-0.131754	0.295688	-0.092061	-0.127291	0.457030	1.000000	0.369152	0.476002	0.053291	0.580328	-0.119746	-0.126632
primary_cleaner.output.concentrate_pb	0.204538	-0.045795	0.306172	0.129691	0.032741	-0.051920	0.368670	-0.152814	0.458708	0.369152	1.000000	0.377527	0.312420	0.090878	0.368866	0.020953
final.output.concentrate_pb	-0.079812	-0.030951	0.010703	0.374400	-0.207999	-0.253089	-0.036563	0.063570	0.301446	0.476002	0.377527	1.000000	0.116718	0.020950	-0.110350	-0.048586
rougher.input.feed_sol	0.094392	-0.090094	-0.091333	0.219287	0.115878	0.159997	0.232016	-0.036783	0.090636	0.053291	0.312420	0.116718	1.000000	0.103491	0.178899	0.010362
rougher.output.concentrate_sol	0.221388	0.868293	0.061587	0.083825	0.137356	0.685685	0.049560	-0.018497	0.191396	0.580328	0.090878	0.020950	0.103491	1.000000	0.050066	0.183095
primary_cleaner.output.concentrate_sol	0.476985	0.011100	0.165738	-0.006042	0.553543	0.301270	0.724903	0.332771	0.185224	-0.119746	0.368866	-0.110350	0.178899	0.050066	1.000000	0.240549
final.output.concentrate_sol	0.474520	0.225260	-0.054210	0.261225	0.479259	0.359009	0.362700	0.380142	0.185576	-0.126632	0.020953	-0.048586	0.010362	0.183095	0.240549	1.000000

	rougher.input.feed_au	rougher.input.feed_ag	rougher.input.feed_pb	rougher.input.feed_sol	rougher.input.feed_total
date
2018-03-12 02:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-03-12 03:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-03-12 04:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-03-12 05:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-03-12 06:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2016-09-02 06:59:59	0.568632	0.568441	0.273677	3.091118	4.501868
2017-11-04 23:59:59	2.764981	2.667444	1.246564	1.393488	8.072476
2018-03-15 16:59:59	1.265676	1.226494	0.613734	5.112079	8.217984
2016-03-13 12:00:00	1.131434	1.106758	0.358522	5.622568	8.219283
2016-04-21 00:00:00	1.033221	1.116930	0.410705	5.868278	8.429133
2017-08-13 17:59:59	1.057161	1.015676	0.609395	6.785796	9.468028
2018-05-15 04:59:59	1.737032	1.778221	0.575469	5.423680	9.514401

	rougher.output.concentrate_au	rougher.output.concentrate_ag	rougher.output.concentrate_pb	rougher.output.concentrate_sol	rougher.output.concentrate_total
date
2016-01-16 21:00:00	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-19 21:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-19 20:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-19 19:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-19 18:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
...	...	...	...	...	...
2017-04-07 07:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-04-07 06:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-04-27 17:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-07-26 23:59:59	1.889855	0.905993	0.629964	2.152051	5.577864
2018-03-12 01:59:59	2.851655	1.632650	0.868842	4.092777	9.445924

	primary_cleaner.output.concentrate_au	primary_cleaner.output.concentrate_ag	primary_cleaner.output.concentrate_pb	primary_cleaner.output.concentrate_sol	primary_cleaner.output.concentrate_total
date
2016-01-15 19:00:00	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-06 19:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-10-18 12:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-10-07 10:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-09-26 12:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-08-18 20:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-08-18 19:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-08-18 18:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-08-18 17:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-08-18 16:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-08-12 03:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-08-12 01:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-08-12 00:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-08-11 23:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-08-11 22:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-08-11 21:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-08-11 20:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-08-11 19:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-08-11 18:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-08-11 17:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-08-11 16:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-08-11 15:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-07-25 14:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-07-18 21:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-07-13 05:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-07-08 09:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-17 19:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-17 20:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-19 13:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-19 14:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2018-02-08 15:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2018-01-31 12:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2018-01-30 21:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2018-01-30 18:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-12-30 08:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-12-30 05:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-12-24 08:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-12-15 09:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-12-15 08:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-12-15 07:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-12-15 06:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-12-13 17:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-06-08 14:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-12-13 16:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-12-01 00:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-30 23:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-30 22:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-30 21:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-30 11:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-30 09:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-30 08:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-28 14:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-28 13:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-28 12:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-27 20:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-19 15:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-12-01 01:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-06-08 13:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2018-02-03 01:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-12-21 15:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-01-16 06:00:00	0.000000	0.000000	0.000000	0.000000	0.000000
2017-01-19 00:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-01-06 23:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-01-06 22:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-01-18 01:00:00	0.000000	0.000000	0.000000	0.000000	0.000000
2017-01-06 21:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-01-06 20:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-01-06 19:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-11-28 18:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-08-23 15:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-02-17 16:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-07-06 13:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-04-10 23:00:00	0.000000	0.000000	0.000000	0.000000	0.000000
2016-04-10 13:00:00	0.000000	0.000000	0.000000	0.000000	0.000000
2016-03-20 17:00:00	0.000000	0.000000	0.000000	0.000000	0.000000
2016-01-18 04:00:00	0.000000	0.000000	0.000000	0.000000	0.000000
2016-02-18 09:00:00	0.000000	0.000000	0.000000	0.000000	0.000000
2016-02-18 08:00:00	0.000000	0.000000	0.000000	0.000000	0.000000
2016-01-20 17:00:00	0.000000	0.000000	0.000000	0.000000	0.000000
2016-01-18 20:00:00	0.000000	0.000000	0.000000	0.000000	0.000000
2016-01-18 19:00:00	0.000000	0.000000	0.000000	0.000000	0.000000
2017-06-08 12:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-04-16 08:00:00	0.000000	0.000000	0.000000	0.000000	0.000000
2017-02-17 17:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-01-15 22:00:00	0.000000	0.000000	0.000000	0.000000	0.000000
2017-04-09 10:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-04-11 18:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-04-09 15:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-04-09 14:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-02-17 18:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-04-09 13:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-04-09 12:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-04-09 11:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-04-11 19:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-04-12 02:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-04-03 19:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-04-15 08:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-02-24 01:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-02-21 11:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-04-23 20:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-04-23 21:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-02-17 19:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-04-15 07:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2018-06-26 23:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-06-26 18:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-04 10:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-06-02 12:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-04 11:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-06-26 10:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-04 12:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-06-26 07:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-06-25 09:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-06-02 11:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-06-13 13:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-04 09:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-06-26 17:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-04 08:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-02 03:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-04 02:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-03 10:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-03 11:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-03 12:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-03 13:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-03 14:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-03 15:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-03 16:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-03 17:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-03 18:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-03 19:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-03 20:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-03 21:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-06-28 10:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-03 22:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-03 23:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-04 00:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-04 01:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-04 07:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-06-28 20:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-08-09 08:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-06-28 22:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-08-09 12:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-08-09 13:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-08-09 15:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-08-09 16:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-08-09 17:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-08-09 18:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-08-09 19:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-08-09 22:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-08-09 23:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-08-10 00:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-08-10 01:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-08-10 02:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-08-10 03:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-08-10 04:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-08-10 05:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-08-09 11:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-06-28 21:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-08-09 10:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-03 09:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-06-28 23:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-06-29 00:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-06-29 01:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-06-29 02:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-06-29 13:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-06-29 14:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-06-29 15:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-07-13 20:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-07-13 21:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-07-28 05:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-07-28 06:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-08-05 01:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-08-09 05:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-08-09 06:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-08-09 07:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-08-09 09:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-03 08:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-08-10 07:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-03 06:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-03-25 18:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-03-25 17:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-03-14 00:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-03-13 23:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-03-03 19:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-03-03 18:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-03-03 17:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-03-03 16:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-01 17:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-03-03 15:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-03-01 15:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-03 07:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-02-22 07:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-02-21 14:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-02-17 19:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-02-17 18:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-02-17 17:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-08-10 06:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-03-02 08:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-01 21:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-02-22 22:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-01 23:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-01 22:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-03 04:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-03 00:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-02 23:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-02 22:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-02 21:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-02 19:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-02 18:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-02 17:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-02 10:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-02 09:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-03 05:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-02 07:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-02 06:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-02 05:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-02 04:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-02 02:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-02 01:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-02 08:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2018-05-02 00:59:59	0.010000	0.010000	0.010000	0.010000	0.040000
2017-07-13 04:59:59	0.056590	0.018661	0.021357	0.016416	0.113023
2018-07-28 04:59:59	0.135919	0.038086	0.045857	0.062800	0.282661
2017-04-12 01:59:59	0.439876	0.131111	0.135342	0.067671	0.774000
2018-07-04 13:59:59	0.567017	0.116097	0.237962	0.162346	1.083422
2017-02-21 08:59:59	0.737878	0.221596	0.214927	0.139196	1.313597
2016-09-29 13:59:59	0.543813	0.274310	0.247621	0.353542	1.419286
2017-04-11 17:59:59	0.936814	0.258513	0.265159	0.099622	1.560108
2017-12-11 20:59:59	0.942294	0.291130	0.333714	0.381574	1.948711
2017-12-30 07:59:59	1.004621	0.259923	0.323869	0.377467	1.965880
2017-04-12 03:59:59	1.310691	0.334812	0.357572	0.118244	2.121318
2018-02-08 16:59:59	1.466296	0.232558	0.366235	0.514679	2.579767
2018-06-28 11:59:59	1.414231	0.364627	0.415089	0.454591	2.648538
2017-11-03 05:59:59	1.368460	0.388146	0.533317	0.511794	2.801717
2017-11-22 10:59:59	1.277551	0.412264	0.550238	0.573682	2.813734
2017-12-30 10:59:59	1.451441	0.532823	0.563730	0.763511	3.311504
2017-12-17 02:59:59	2.580535	0.402895	0.258659	0.112066	3.354154
2016-03-13 11:00:00	2.091185	0.494574	0.405509	0.789790	3.781057
2018-02-17 16:59:59	1.961528	0.714743	0.620558	0.859003	4.155832
2018-04-26 18:59:59	2.206751	0.720534	0.881293	0.761659	4.570236
2017-04-03 20:59:59	2.826480	0.615718	0.765910	0.423060	4.631168
2017-12-30 04:59:59	2.304869	0.646440	0.778103	0.926534	4.655947
2018-08-09 03:59:59	2.512162	0.693948	0.761075	0.785608	4.752794
2016-01-18 18:00:00	2.484481	0.484025	0.675310	1.116411	4.760227
2018-06-12 15:59:59	2.727204	0.593382	1.042643	0.664624	5.027853
2018-01-30 19:59:59	3.153500	0.671739	0.895703	0.977015	5.697957
2018-01-16 16:59:59	2.886705	0.799691	1.046693	1.195250	5.928339
2017-01-18 13:59:59	4.265764	0.765107	0.791542	0.121220	5.943633
2018-06-27 00:59:59	3.517463	0.898312	0.967847	1.222345	6.605967
2017-03-08 01:59:59	3.901514	0.852267	1.067738	0.882269	6.703789
2016-01-18 07:00:00	3.660422	0.724508	0.938667	1.736741	7.060338
2018-02-03 02:59:59	2.991256	1.126822	1.254637	1.758659	7.131374
2017-05-09 07:59:59	3.943262	1.259338	1.315968	0.938361	7.456929
2016-01-15 21:00:00	3.427319	0.954876	1.137228	1.968954	7.488378
2017-02-16 20:59:59	3.916086	1.319489	1.501590	1.410451	8.147616
2018-06-14 01:59:59	4.697498	1.012395	1.615224	1.118362	8.443479
2017-12-24 18:59:59	4.116963	1.446499	1.736333	2.083193	9.382989
2017-10-08 19:59:59	5.217559	1.274083	1.784698	1.422342	9.698682
2017-12-15 10:59:59	6.336979	0.885246	1.194370	1.313387	9.729983

	final.output.concentrate_au	final.output.concentrate_ag	final.output.concentrate_pb	final.output.concentrate_sol	final.output.concentrate_total
date
2016-02-13 11:00:00	0.000000	0.000000	0.000000	0.000000	0.000000
2017-10-17 22:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-10-13 13:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-10-12 20:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-10-12 14:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-10-07 05:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-09-26 13:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-09-11 13:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-09-11 12:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-09-08 16:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-09-07 22:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-07-26 04:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-07-26 03:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-07-25 08:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-07-25 07:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-07-24 13:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-07-23 23:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-07-14 09:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-05-23 19:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-05-20 23:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-10-21 02:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-05-20 22:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-10-21 03:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-05 03:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2018-01-15 21:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2018-01-15 20:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-12-26 20:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-12-26 19:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-12-25 20:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-12-17 12:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-12-17 11:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-12-17 08:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-12-17 04:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-12-17 03:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-12-17 01:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-12-17 00:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-12-16 23:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-12-12 05:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-30 02:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-30 01:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-29 23:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-18 08:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-18 07:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-11-01 11:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2018-01-25 10:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-05-09 04:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-04-23 18:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-11-01 18:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-09-29 14:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-09-14 17:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-08-30 19:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-08-30 18:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-08-26 20:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-08-15 23:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-08-11 20:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-07-06 12:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-11-19 15:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-07-06 08:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-07-06 05:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-07-06 04:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-05-11 01:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-05-02 00:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-05-01 23:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-03-11 13:00:00	0.000000	0.000000	0.000000	0.000000	0.000000
2016-03-11 12:00:00	0.000000	0.000000	0.000000	0.000000	0.000000
2016-03-11 11:00:00	0.000000	0.000000	0.000000	0.000000	0.000000
2016-03-11 10:00:00	0.000000	0.000000	0.000000	0.000000	0.000000
2016-07-06 06:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-04-23 19:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2016-11-26 19:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-01-02 17:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-04-23 17:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-04-23 16:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-04-23 15:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-03-28 02:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-03-17 10:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-03-06 20:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-03-06 19:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-03-06 18:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-03-06 17:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-01-02 16:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-02-28 15:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-02-28 14:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-02-24 03:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-01-26 10:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-01-26 09:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-01-18 20:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-01-18 16:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-01-18 15:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-01-18 14:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2017-01-02 18:59:59	0.000000	0.000000	0.000000	0.000000	0.000000
2018-08-08 04:59:59	1.063992	0.289059	0.332391	0.291099	1.976540
2017-09-07 21:59:59	1.195223	0.193950	0.571758	0.268995	2.229927
2017-12-17 10:59:59	0.904254	2.876804	2.877504	2.645650	9.304212

	params	mean_test_score	rank_test_score	mean_fit_time	mean_score_time
34	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.069732	1	1.638173	0.007624
20	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.069751	2	1.670194	0.006691
62	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.069751	3	1.632306	0.005472
6	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.069753	4	1.636373	0.007387
41	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.069755	5	1.637281	0.006745
27	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.069756	6	1.675049	0.007347
55	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.069756	7	1.600286	0.008911
13	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.069757	8	1.709086	0.006086
69	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.069759	9	1.627007	0.007118
48	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.069760	10	1.586270	0.005321
33	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.070549	11	0.171437	0.023949
40	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.070549	11	0.175679	0.039749
19	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.070567	13	0.322410	0.049916
26	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.070567	13	0.297921	0.038491
68	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.070711	15	0.192978	0.005821
61	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.070711	15	0.192583	0.006094
54	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.070725	17	0.213103	0.006090
47	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.070725	17	0.230960	0.006490
12	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.070797	19	0.382653	0.039399
5	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.070797	19	0.268508	0.044162
65	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071177	21	0.337505	0.077886
58	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071177	21	0.242490	0.074002
59	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071177	23	0.120475	0.039606
66	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071177	23	0.136015	0.023483
64	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071177	23	0.141518	0.022923
57	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071177	23	0.154103	0.023805
51	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071199	27	0.274510	0.043794
44	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071199	27	0.292685	0.062716
45	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071199	29	0.138886	0.039778
43	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071199	29	0.124517	0.042792
52	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071199	29	0.118993	0.039446
50	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071199	29	0.158384	0.023449
30	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071220	33	0.224325	0.078599
37	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071220	33	0.238495	0.040787
31	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071220	35	0.118852	0.023554
36	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071220	35	0.118819	0.039593
38	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071220	35	0.134973	0.022118
29	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071220	35	0.117892	0.036698
16	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071242	39	0.553139	0.062917
23	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071242	39	0.294752	0.046650
17	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071242	41	0.102572	0.039317
24	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071242	41	0.114532	0.040849
22	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071242	41	0.117981	0.026494
15	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071242	41	0.144293	0.031562
2	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071264	45	0.455105	0.061591
9	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071264	45	0.606112	0.039530
3	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071264	47	0.077358	0.020287
10	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071264	47	0.109989	0.010197
1	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071264	47	0.094002	0.027350
8	{'regressor': Ridge(alpha=0.6, solver='sag'), ...	-5.071264	47	0.109646	0.060326

	params	mean_test_score	rank_test_score	mean_fit_time	mean_score_time
89	{'regressor': RandomForestRegressor(bootstrap=...	-3.364550	1	0.341438	0.006719
88	{'regressor': RandomForestRegressor(bootstrap=...	-3.383517	2	0.315439	0.006786
87	{'regressor': RandomForestRegressor(bootstrap=...	-3.388674	3	0.281886	0.006965
86	{'regressor': RandomForestRegressor(bootstrap=...	-3.399301	4	0.243524	0.006391
85	{'regressor': RandomForestRegressor(bootstrap=...	-3.413911	5	0.214493	0.005753
84	{'regressor': RandomForestRegressor(bootstrap=...	-3.428317	6	0.177909	0.005780
83	{'regressor': RandomForestRegressor(bootstrap=...	-3.480400	7	0.149612	0.005539
80	{'regressor': RandomForestRegressor(bootstrap=...	-3.535932	8	0.326547	0.007013
78	{'regressor': RandomForestRegressor(bootstrap=...	-3.546513	9	0.259711	0.006045
79	{'regressor': RandomForestRegressor(bootstrap=...	-3.553310	10	0.293584	0.006433
82	{'regressor': RandomForestRegressor(bootstrap=...	-3.553318	11	0.107843	0.005093
77	{'regressor': RandomForestRegressor(bootstrap=...	-3.560138	12	0.222078	0.006002
75	{'regressor': RandomForestRegressor(bootstrap=...	-3.572244	13	0.162701	0.005691
76	{'regressor': RandomForestRegressor(bootstrap=...	-3.578308	14	0.196820	0.005755
74	{'regressor': RandomForestRegressor(bootstrap=...	-3.606519	15	0.130206	0.005266
81	{'regressor': RandomForestRegressor(bootstrap=...	-3.673081	16	0.084222	0.004980
71	{'regressor': RandomForestRegressor(bootstrap=...	-3.697908	17	0.285954	0.006277
73	{'regressor': RandomForestRegressor(bootstrap=...	-3.708031	18	0.098550	0.005116
70	{'regressor': RandomForestRegressor(bootstrap=...	-3.717101	19	0.258681	0.006025
69	{'regressor': RandomForestRegressor(bootstrap=...	-3.727518	20	0.237331	0.006303
68	{'regressor': RandomForestRegressor(bootstrap=...	-3.729483	21	0.217456	0.006467
67	{'regressor': RandomForestRegressor(bootstrap=...	-3.749426	22	0.176434	0.005463
66	{'regressor': RandomForestRegressor(bootstrap=...	-3.754197	23	0.147869	0.005380
72	{'regressor': RandomForestRegressor(bootstrap=...	-3.774085	24	0.069421	0.004748
65	{'regressor': RandomForestRegressor(bootstrap=...	-3.815373	25	0.120864	0.005758
64	{'regressor': RandomForestRegressor(bootstrap=...	-3.905630	26	0.093602	0.005053
62	{'regressor': RandomForestRegressor(bootstrap=...	-3.926760	27	0.258277	0.005917
8	{'regressor': DecisionTreeRegressor(random_sta...	-3.933135	28	0.039105	0.003645
61	{'regressor': RandomForestRegressor(bootstrap=...	-3.948316	29	0.235572	0.005827
60	{'regressor': RandomForestRegressor(bootstrap=...	-3.954834	30	0.207015	0.006205
59	{'regressor': RandomForestRegressor(bootstrap=...	-3.967227	31	0.184797	0.005456
63	{'regressor': RandomForestRegressor(bootstrap=...	-3.971902	32	0.073915	0.005663
58	{'regressor': RandomForestRegressor(bootstrap=...	-3.972317	33	0.164251	0.005707
57	{'regressor': RandomForestRegressor(bootstrap=...	-3.974593	34	0.133096	0.005053
56	{'regressor': RandomForestRegressor(bootstrap=...	-4.010730	35	0.112591	0.005252
7	{'regressor': DecisionTreeRegressor(random_sta...	-4.031044	36	0.036608	0.003769
55	{'regressor': RandomForestRegressor(bootstrap=...	-4.088993	37	0.087396	0.005056
54	{'regressor': RandomForestRegressor(bootstrap=...	-4.130041	38	0.059482	0.004572
53	{'regressor': RandomForestRegressor(bootstrap=...	-4.164852	39	0.240518	0.006165
52	{'regressor': RandomForestRegressor(bootstrap=...	-4.180443	40	0.234072	0.006658
51	{'regressor': RandomForestRegressor(bootstrap=...	-4.184709	41	0.196469	0.005529
48	{'regressor': RandomForestRegressor(bootstrap=...	-4.196637	42	0.128139	0.005680
50	{'regressor': RandomForestRegressor(bootstrap=...	-4.201636	43	0.165721	0.005604
49	{'regressor': RandomForestRegressor(bootstrap=...	-4.210206	44	0.158522	0.005708
47	{'regressor': RandomForestRegressor(bootstrap=...	-4.231113	45	0.107984	0.005509
46	{'regressor': RandomForestRegressor(bootstrap=...	-4.315249	46	0.081785	0.004996
6	{'regressor': DecisionTreeRegressor(random_sta...	-4.326118	47	0.035381	0.004336
44	{'regressor': RandomForestRegressor(bootstrap=...	-4.409566	48	0.205687	0.005875
45	{'regressor': RandomForestRegressor(bootstrap=...	-4.420534	49	0.061058	0.005212
42	{'regressor': RandomForestRegressor(bootstrap=...	-4.435392	50	0.160894	0.005467

	params	mean_test_score	rank_test_score	mean_fit_time	mean_score_time
60	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.940929	1	0.277492	0.039933
67	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.940929	1	0.257478	0.026162
46	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.940943	3	0.273829	0.025581
53	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.940943	3	0.313441	0.046658
32	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.940957	5	0.242047	0.057875
39	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.940957	5	0.264061	0.055793
18	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.940972	7	0.229300	0.069559
25	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.940972	7	0.263371	0.076565
4	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.940986	9	0.225335	0.053766
11	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.940986	9	0.280671	0.058038
68	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942620	11	0.298271	0.060502
61	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942620	11	0.317063	0.041974
54	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942724	13	0.560341	0.058916
47	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942724	13	0.299800	0.076980
65	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942773	15	0.439666	0.078893
58	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942773	15	0.456932	0.061961
57	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942773	17	0.121743	0.148678
66	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942773	17	0.175463	0.087964
59	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942773	17	0.173659	0.125976
64	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942773	17	0.181529	0.126593
26	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942782	21	0.279208	0.078217
19	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942782	21	0.278022	0.060654
44	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942805	23	0.455988	0.062686
51	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942805	23	0.477897	0.080293
52	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942805	25	0.258209	0.081121
45	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942805	25	0.313809	0.025848
43	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942805	25	0.223115	0.177434
50	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942805	25	0.199601	0.077992
5	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942809	29	0.301202	0.077733
12	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942809	29	0.300098	0.040412
37	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942837	31	0.399380	0.079655
30	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942837	31	0.500719	0.079050
38	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942837	33	0.178561	0.099856
29	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942837	33	0.282170	0.096002
31	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942837	33	0.257594	0.101328
36	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942837	33	0.181655	0.157988
23	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942870	37	0.438475	0.079215
16	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942870	37	0.421228	0.080212
24	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942870	39	0.174323	0.124263
15	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942870	39	0.233925	0.117019
17	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942870	39	0.239587	0.118842
22	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942870	39	0.126629	0.164403
9	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942903	43	0.440812	0.078028
2	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942903	43	0.718790	0.080670
10	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942903	45	0.158031	0.119888
1	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942903	45	0.205478	0.115996
8	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942903	45	0.212418	0.111885
3	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.942903	45	0.189126	0.127898
33	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.943016	49	0.278549	0.077032
40	{'regressor': Ridge(alpha=1, solver='lsqr'), '...	-7.943016	49	0.381297	0.077657

	params	mean_test_score	rank_test_score	mean_fit_time	mean_score_time
88	{'regressor': RandomForestRegressor(bootstrap=...	-6.410721	1	0.529923	0.007821
89	{'regressor': RandomForestRegressor(bootstrap=...	-6.412185	2	0.592436	0.008017
87	{'regressor': RandomForestRegressor(bootstrap=...	-6.414666	3	0.464874	0.007480
86	{'regressor': RandomForestRegressor(bootstrap=...	-6.429849	4	0.408293	0.007053
85	{'regressor': RandomForestRegressor(bootstrap=...	-6.467907	5	0.358140	0.006888
84	{'regressor': RandomForestRegressor(bootstrap=...	-6.496217	6	0.295741	0.006471
80	{'regressor': RandomForestRegressor(bootstrap=...	-6.522729	7	0.535046	0.007757
79	{'regressor': RandomForestRegressor(bootstrap=...	-6.539747	8	0.488436	0.007245
78	{'regressor': RandomForestRegressor(bootstrap=...	-6.542670	9	0.437712	0.007369
83	{'regressor': RandomForestRegressor(bootstrap=...	-6.544804	10	0.246835	0.006379
77	{'regressor': RandomForestRegressor(bootstrap=...	-6.568959	11	0.386308	0.007094
76	{'regressor': RandomForestRegressor(bootstrap=...	-6.612900	12	0.346229	0.006992
75	{'regressor': RandomForestRegressor(bootstrap=...	-6.651695	13	0.288483	0.006748
82	{'regressor': RandomForestRegressor(bootstrap=...	-6.662844	14	0.182981	0.005897
74	{'regressor': RandomForestRegressor(bootstrap=...	-6.713134	15	0.231579	0.006739
71	{'regressor': RandomForestRegressor(bootstrap=...	-6.752664	16	0.494115	0.007222
70	{'regressor': RandomForestRegressor(bootstrap=...	-6.756856	17	0.442874	0.006759
69	{'regressor': RandomForestRegressor(bootstrap=...	-6.774849	18	0.395398	0.006620
68	{'regressor': RandomForestRegressor(bootstrap=...	-6.790953	19	0.352791	0.006720
67	{'regressor': RandomForestRegressor(bootstrap=...	-6.811870	20	0.314630	0.007784
66	{'regressor': RandomForestRegressor(bootstrap=...	-6.835059	21	0.248420	0.006260
73	{'regressor': RandomForestRegressor(bootstrap=...	-6.835775	22	0.168864	0.006104
65	{'regressor': RandomForestRegressor(bootstrap=...	-6.879913	23	0.201483	0.005853
81	{'regressor': RandomForestRegressor(bootstrap=...	-6.933152	24	0.130295	0.005919
61	{'regressor': RandomForestRegressor(bootstrap=...	-6.957991	25	0.397876	0.006703
62	{'regressor': RandomForestRegressor(bootstrap=...	-6.959342	26	0.433555	0.006977
60	{'regressor': RandomForestRegressor(bootstrap=...	-6.978162	27	0.368637	0.006581
59	{'regressor': RandomForestRegressor(bootstrap=...	-7.007841	28	0.319525	0.006424
64	{'regressor': RandomForestRegressor(bootstrap=...	-7.009522	29	0.154379	0.005879
58	{'regressor': RandomForestRegressor(bootstrap=...	-7.027324	30	0.289116	0.006777
72	{'regressor': RandomForestRegressor(bootstrap=...	-7.047439	31	0.131548	0.006016
57	{'regressor': RandomForestRegressor(bootstrap=...	-7.069723	32	0.228759	0.006252
56	{'regressor': RandomForestRegressor(bootstrap=...	-7.100645	33	0.188819	0.006118
55	{'regressor': RandomForestRegressor(bootstrap=...	-7.183593	34	0.142257	0.005673
53	{'regressor': RandomForestRegressor(bootstrap=...	-7.224745	35	0.383275	0.006617
51	{'regressor': RandomForestRegressor(bootstrap=...	-7.226765	36	0.314697	0.006412
52	{'regressor': RandomForestRegressor(bootstrap=...	-7.231315	37	0.354456	0.006317
50	{'regressor': RandomForestRegressor(bootstrap=...	-7.236839	38	0.274154	0.006163
63	{'regressor': RandomForestRegressor(bootstrap=...	-7.264018	39	0.109458	0.005619
49	{'regressor': RandomForestRegressor(bootstrap=...	-7.271231	40	0.238654	0.006205
48	{'regressor': RandomForestRegressor(bootstrap=...	-7.287323	41	0.196122	0.005891
47	{'regressor': RandomForestRegressor(bootstrap=...	-7.332006	42	0.162587	0.005684
54	{'regressor': RandomForestRegressor(bootstrap=...	-7.372428	43	0.112661	0.006777
46	{'regressor': RandomForestRegressor(bootstrap=...	-7.410342	44	0.123577	0.005576
44	{'regressor': RandomForestRegressor(bootstrap=...	-7.523149	45	0.333321	0.006564
43	{'regressor': RandomForestRegressor(bootstrap=...	-7.526228	46	0.304654	0.006462
42	{'regressor': RandomForestRegressor(bootstrap=...	-7.536112	47	0.266169	0.006057
41	{'regressor': RandomForestRegressor(bootstrap=...	-7.550396	48	0.232282	0.005832
8	{'regressor': DecisionTreeRegressor(random_sta...	-7.556558	49	0.069121	0.005008
40	{'regressor': RandomForestRegressor(bootstrap=...	-7.558464	50	0.205469	0.005769

Восстановление золота из руды

Восстановление золота из руды¶

Введение¶

Библиотеки и настройки¶

Библиотеки¶

Настройки¶

Подготовка данных¶

Загрузка данных¶

Индексы¶

Знакомство с данными¶

Описание датафреймов¶

Колонки¶

Пропуски¶

Выводы:¶

rougher.output.recovery¶

Выводы:¶

final.output.recovery¶

Выводы:¶

Предобработка данных¶

Индексы¶

Пропуски¶

Выводы:¶

Анализ данных¶

Концентрация металлов (Au, Ag, Pb) на различных этапах очистки.¶

Au¶

Ag¶

Pb¶

Sol¶

Визуализация¶

Выводы:¶

Cуммарная концентрация веществ на разных стадиях¶

Визуализация¶

Околонулевые значения¶

Золотоносная руда¶

Черновой концентрат¶

Концентрат после первой очистки¶

Финальный концентрат¶

Визуализация¶

Выводы:¶

Распределения размеров гранул на обучающей и тестовой выборках¶

rougher.input.feed_size¶

Графики для размеров гранул перед флотацией в train и test¶

Статистика для размеров гранул перед флотацией в train и test¶

train¶

test¶

Выводы и Переформулирование задачи¶

Доверительные интервалы¶

Вывод:¶

primary_cleaner.input.feed_size¶

Графики для размеров гранул после флотацией/перед первой очисткой в train и test¶

Статистика для размеров гранул после флотации/перед первой очисткой в train и test¶

train¶

test¶

Вывод:¶

Доверительные интервалы¶

Вывод:¶

Вывод:¶

Выводы:¶

Модели¶

Генератор псевдослучайных значений¶

Целевой признак и обучающие данные¶

Разбиение данных на выборки¶

rougher.output.recovery пары для обучения/валидации и теста¶

для обучения/валидации¶

для теста¶

final.output.recovery пары для обучения/валидации и теста¶

для обучения/валидации¶

для теста¶

Кросс-валидация¶

Метрики (sMAPE, Итоговый sMAPE)¶

Границы адекватности¶

sMAPE¶

для rougher (флотации)¶

для final (всего технологического процесса)¶

Итоговая sMAPE¶

Выводы:¶

Модели¶

Преподготовка данных¶

Шаги по умолчанию для pipeline для GridSearchCV¶

Модели и их параметры для выбора лучшей в GridSearchCV¶